How to Build and Deploy Docker Images on GKE (Google Kubernetes Engine)— Part 1
This blog series covers the steps involved in building a Docker image to deploy and run it using Google Kubernetes Engine (GKE). Should you want to deploy on another public cloud, the concepts provided below will be somewhat similar.
I will demonstrate this process using a series of simple steps with less technical and complicated jargon than what you might ordinarily find.
In my previous blog post, I walked through how Python is used for running basic Snowflake commands (CRUD). It’s a good starting point for anyone who wants to utilize Python for Snowflake, so be sure to check that out later on!
Here’s what you can expect
I’ve broken the process down into two parts (and posts):
PART 1
Step 1: Create a simple Python script that will be dockerized or containerized in an image. This may be a Python script or any other code; the steps will be the same. Ensure what you want to build is running successfully on your local machine and be aware of the dependencies.
Step 2: Build and verify the Docker image.
(Skip step 1 and 2 if you already have a working Docker image)
PART 2
Step 3: Set up GKE and deploy the Docker image on it.
Read the second blog post in this series here: GCP Kubernetes.
Prerequisite: This is a fairly simple task. A basic familiarity with Docker, Kubernetes, or any cloud service provider will be sufficient.
Level: Beginner
Resources:
- Github → https://github.com/architpandita/sample-app
- GKE → https://cloud.google.com/kubernetes-engine/docs/tutorials/hello-app
- Docker Setup → https://docs.docker.com/get-started/
- Kubernetes → https://kubernetes.io/docs/tutorials/kubernetes-basics/
Let’s Begin! PART 1
Step 1: Create a Working Application
I’m using a Python script that reads the JSON file and stores it as CSV.
( Python 3.6+ and PyCharm)
App Structure
Sample→ directory which contains all the applications
Sample.json → JSON file which is used as a source
Sample.csv → target filename which will be stored in the “CSV” directory
json_csv.py → Python script to convert JSON to CSV
After running the script:
Create a “requirements.txt” file
Create a “requirements.txt” file
Let’s create a requirements file for required Python dependencies based on Python env:
Run in the terminal of PyCharm inside the sample directory:
“ pip freeze > requirement.txt ”
This will create the requirement.txt file containing all the dependencies with their respective versions.
Step 2: Build the Docker image
Docker is more like the packaging of all the application-related files. Code including the operating system and this package are supposed to have the bare minimum of things required for the application to run. This keeps Docker lightweight and efficient.
Like a bootable pen drive, plug n play.
Think of it like (not an exact analogy, but you will get an idea) having to make a virtual environment that contains your application allowing you to use the virtual environment to run your application on any platform.
To set up that environment, you will have to install a few software dependencies. For example, to run Python, we must have Python installed. Also, we have some local files that we want to put inside the environment as well. Although it is a closed environment, you can still use “ docker expose” to open some external connection ports.
To do this, we must give Docker instructions for building the environment using a Dockerfile.
( If you face a certification failure error, use the line below:
RUN pip install — trusted-host pypi.python.org — trusted-host pypi.org — trusted-host files.pythonhosted.org -r requirements.txt )
Go to the folder where you have “Dockerfile.” In this example, we will use the “sample” folder.
run cmd “ docker build -t sample-app . ”
“-t” is for a tag. You can set different versioning for your build. “ . ” is used when Dockerfile is present at the current location.
For example: “ docker build -t sample-app:1.0 ”
Tip: Try not to use ”_” as GCP won’t allow this naming convention in some services. Using “-” is a safer solution for the naming convention.
This will build the Docker image, as you can see below. It is building each instruction as given in Dockerfile.
Make sure the path and file mention are reachable.
Note: I have already built this so that you will see fewer lines in CMD. For a new build, there will be more lines, such as installation details.
Run cmd “ docker image ls ”
This will show all the available images, as you can see below. Since we have not given the tag, the default is “latest.” Also, check the “ ImageID,” which is used for logs and status.
Now, verify if your build has successfully run:
Run cmd “ docker run sample-app:latest”
If you are successful, it runs without error and displays the expected output.
Now, we have successfully built the Docker image.
Use “ docker ps ” to check if an image is currently running. Use “Image Id” to check which image is running.
Step 3: Deploying the Docker image on GKE
I hope this gets you started. Be sure and check out Part 2 of this series, where I explore the third step, “How to Deploy on GKE.” You can read that article here.
At the start, my goal was to make this a simple and easy-to-follow quick start guide. Please let me know your thoughts and any comments — it would be great to have feedback from the community.
Ready to Accelerate Your Digital Transformation?
At Hashmap, we work with our clients to build better together.
If you consider moving data and analytics products and applications to the cloud or if you would like help and guidance and a few best practices in delivering higher value outcomes in your existing cloud program, please contact us.
Hashmap, an NTT DATA Company, offers a range of enablement workshops and assessment services, cloud modernization and migration services, and consulting service packages as part of our data and cloud service offerings. We would be glad to work through your specific requirements.
Other Tools and Content You Might Like
Feel free to share on other channels, and be sure and keep up with all new content from Hashmap here. To listen in on a casual conversation about all things data engineering and the cloud, check out Hashmap’s podcast Hashmap on Tap as well on Spotify, Apple, Google, and other popular streaming apps.
Archit Pandita is a Python developer and Data Engineer with Hashmap, an NTT DATA Company, providing Data, Cloud, IoT, and AI/ML solutions and expertise in the financial domain with a group of innovative technologists and domain experts accelerating high-value business outcomes for our customers. Connect with him on LinkedIn.
