--- title: Automate python libraries deployment at AWS Lambda layer from Pipfile with Terraform - Best Software Development Team Extension Partner for Nordics description: AWS Lambda, a part of Amazon Web Services (AWS) is a serverless computing service that works as FaaS (Function as a Service). A FaaS is a service which image: https://www.craftsmensoftware.com/hubfs/Imported_Blog_Media/Automate_Python_Libraries_Deployment_At_AWS_Lambda_Layer.webp --- [Skip to main content](https://www.craftsmensoftware.com/blog/automate-python-libraries-deployment-at-aws-lambda-layer-from-pipfile-with-terraform#main-content) [![Craftsmen-logo](https://www.craftsmensoftware.com/hs-fs/hubfs/Website%202.png?width=200&height=72&name=Website%202.png) ![Craftsmen-logo](https://www.craftsmensoftware.com/hs-fs/hubfs/Website%202.png?width=200&height=72&name=Website%202.png) ![Craftsmen-logo](https://www.craftsmensoftware.com/hs-fs/hubfs/Website%202.png?width=200&height=72&name=Website%202.png) ![Craftsmen-logo](https://www.craftsmensoftware.com/hs-fs/hubfs/Website%202.png?width=200&height=72&name=Website%202.png)](https://www.craftsmensoftware.com) - [About us](https://www.craftsmensoftware.com/about-us) - [Client Stories](https://www.craftsmensoftware.com/client-success-stories) - [How We Work](https://www.craftsmensoftware.com/how-we-work) - [Case Study](https://www.craftsmensoftware.com/case-study) - [Blog](https://www.craftsmensoftware.com/blog) - [Career](https://careers.craftsmensoftware.com/) Open main navigation Close main navigation - [About us](https://www.craftsmensoftware.com/about-us) - [Client Stories](https://www.craftsmensoftware.com/client-success-stories) - [How We Work](https://www.craftsmensoftware.com/how-we-work) - [Case Study](https://www.craftsmensoftware.com/case-study) - [Blog](https://www.craftsmensoftware.com/blog) - [Career](https://careers.craftsmensoftware.com/) - [Get In Touch](https://www.craftsmensoftware.com/contact-us) [Get In Touch](https://www.craftsmensoftware.com/contact-us) ## Automate python libraries deployment at AWS Lambda layer from Pipfile with Terraform ![Automate python libraries deployment at AWS Lambda layer from Pipfile with Terraform](https://www.craftsmensoftware.com/hs-fs/hubfs/Imported_Blog_Media/Automate_Python_Libraries_Deployment_At_AWS_Lambda_Layer.webp?width=1200&height=820&name=Automate_Python_Libraries_Deployment_At_AWS_Lambda_Layer.webp) [AWS Lambda](https://aws.amazon.com/lambda), a part of Amazon Web Services (AWS) is a [serverless](https://en.wikipedia.org/wiki/Serverless_computing) computing service that works as [FaaS (Function as a Service)](https://en.wikipedia.org/wiki/Function_as_a_service). A FaaS is a service which provides users to develop and manage application service without thinking about infrastructure. [Terraform](https://www.hashicorp.com/products/terraform) is an Infrastructure as Code ([IaC](https://en.wikipedia.org/wiki/Infrastructure_as_code)) tool which is developed by [Hasi Corp](https://www.hashicorp.com/) to manage resources of cloud services like AWS, Google Cloud, Azure, etc. It is open-source and developed by [golang](https://golang.org/). It is always challenging to zip the codes and upload them for AWS Lambda every time at the time of deployment. The more complex part is to upload codes of libraries e.g python libraries. At [Craftsmen](https://www.craftsmenltd.com/), we need to manage a lot of lambdas for various development purposes. So a smart solution for uploading lambda function code and libraries while deployment is a crying need. Our approach is to upload function codes as function level and libraries in lambda layer. The reasons are 1. Share python libraries between lambdas 2. The console code editor can only visualize [3MB of code](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html). So uploading libraries can make a chance to see codes in console editor. ## Setting the project Let’s start by setting the project skeleton. We are going to use [pipenv](https://docs.pipenv.org/) because it’s more developer-friendly to maintain dev and release packages. First, we install pipenv from [here](https://pipenv-fork.readthedocs.io/en/latest/install.html). Then we will install terraform from [here](https://learn.hashicorp.com/tutorials/terraform/install-cli) `# Create a project directory mkdir lambda-with-terraform cd lambda-with-terraform # Create lambda code directory mkdir lambda # Create Terraform directory mkdir terraform # Add handler.py file in lambda directory touch lambda/handler.py # Add Pipfile in project root directory touch Pipfile # So our skeleton will look like this tree`  Copy   `├── lambda │ └── handler.py ├── Pipfile └── terraform`  Copy Add python libraries We will only use a single library called [**requests**](https://requests.readthedocs.io/)in \[packages\] and pipenv at \[dev-packages\]. Also, we are going to use [**python 3.8**](https://www.python.org/downloads/release/python-380/). Let’s add all to **Pipfile**.  `# Pipfile [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [dev-packages] pytest = "==5.3.5" pipenv = "==2020.8.13" [packages] requests = "==2.23.0" [requires] python_version = "3.8"`  Copy Initiate python virtual environment by pipenv with  `pipenv install`  Copy   This will add the **Pipfile.lock** file which contains information on all python packages. ## Add function code Let’s start with a simple lambda function which just gets a web page and log it.  `# handler.py import requests import logging LOGGER = logging.getLogger() LOGGER.setLevel(logging.INFO) def lambda_handler(event, context): response = requests.get("https://example.com") LOGGER.info(response.text)`  Copy Add Terraform code At first, we will add some files for Terraform  `# Create file for lambda terraform codes touch terraform/lambda.tf # Create file for lambda layer terraform codes touch terraform/lambda_layer.tf # Create python file for make pip requirements file from Pipfile touch terraform/requirements_creator.py # Add shell script to generate pip requirements and make zip file of # lambda libraries touch terraform/build_layer.sh chmod a+x terraform/build_layer.sh`  Copy   In **requirements\_creator.py**, a python argument parser will be added which gets the filename of pip requirements e.g. **requirements.txt**/p\>  `# terraform/requirements_creator.py import argparse from pipenv.project import Project from pipenv.utils import convert_deps_to_pip def _make_requirements_file(file_path): pipfile = Project(chdir=False).parsed_pipfile requirements = convert_deps_to_pip(pipfile['packages'], r=False) with open(file_path, 'w') as req_file: req_file.write('\n'.join(requirements)) def run(): parser = argparse.ArgumentParser() parser.add_argument( '--file_path', '-file_path', type=str, default='requirements.txt' ) args = parser.parse_args() _make_requirements_file(args.file_path) if __name__ == "__main__": run()`  Copy Now let’s add shell script to generate a zip file for lambda layer with python libraries  `# terraform/build_layer.sh DESTINATION_DIR=${DESTINATION_DIR:-$PWD} MODULE_DIR=${MODULE_DIR:-$PWD} ZIPFILE_NAME=${ZIPFILE_NAME:-layer} echo "Module dir $MODULE_DIR" echo "Destination dir $DESTINATION_DIR" TARGET_DIR=$DESTINATION_DIR/$ZIPFILE_NAME echo "Target dir $TARGET_DIR" mkdir -p "$TARGET_DIR" REQUIREMENTS_FILE_PATH=$MODULE_DIR/requirements.txt python3 "$MODULE_DIR"/requirements_creator.py --file_path "$REQUIREMENTS_FILE_PATH" pip install -r "$REQUIREMENTS_FILE_PATH" -t "$TARGET_DIR"/python (cd "$TARGET_DIR" && zip -r "$DESTINATION_DIR"/"$ZIPFILE_NAME".zip ./* -x "*.dist-info*" -x "*__pycache__*" -x "*.egg-info*") rm "$REQUIREMENTS_FILE_PATH" rm -r "$TARGET_DIR"`  Copy Now add lambda layer terraform code  `// terraform/lambda_layer.tf locals { // All lambda codes zip and layer zip file directory lambda_artifact_dir = "${path.module}/lambda_zip" lambda_layer_zipfile_name = "layer" python_version = "python${data.external.python_version.result.version}" } // Grab python version from Pipfile. Default is 3.8 if not mentioned // in Pipfile data "external" "python_version" { program = [ "python3", "-c", "from pipenv.project import Project as P; import json; _p = P(chdir=False); print(json.dumps({'version': _p.required_python_version or '3.8'}))" ] } // Generate zipfile for lambda layer resource "null_resource" "build_lambda_layer" { provisioner "local-exec" { when = create command = "./${path.module}/build_layer.sh" environment = { DESTINATION_DIR = abspath(local.lambda_artifact_dir) MODULE_DIR = abspath(path.module) ZIPFILE_NAME = local.lambda_layer_zipfile_name } } triggers = { // Trigger only when something changes in Pipfile run_on_pipfile_change = filemd5("${abspath(path.module)}/../Pipfile") } } resource "aws_lambda_layer_version" "lambda_layer" { filename = "${local.lambda_artifact_dir}/${local.lambda_layer_zipfile_name}.zip" layer_name = "lambda_layer" compatible_runtimes = [local.python_version] // It will run after lambda layer zipfile build depends_on = [null_resource.build_lambda_layer] lifecycle { create_before_destroy = true } }`  Copy Finally, we are going to add lambda terraform code  `// terraform/lambda.tf // Zip lambda function codes data "archive_file" "lambda_zip_file" { output_path = "${local.lambda_artifact_dir}/lambda.zip" source_dir = "${path.module}/../lambda" excludes = ["__pycache__", "*.pyc"] type = "zip" } data "aws_iam_policy_document" "lambda_assume_role" { version = "2012-10-17" statement { sid = "LambdaAssumeRole" effect = "Allow" actions = [ "sts:AssumeRole" ] principals { identifiers = [ "lambda.amazonaws.com" ] type = "Service" } } } // Lambda IAM role resource "aws_iam_role" "lambda_role" { name = "test-lambda-role" assume_role_policy = data.aws_iam_policy_document.lambda_assume_role.json lifecycle { create_before_destroy = true } } // Lambda function terraform code resource "aws_lambda_function" "lambda_function" { function_name = "test-lambda-function" filename = data.archive_file.lambda_zip_file.output_path source_code_hash = data.archive_file.lambda_zip_file.output_base64sha256 handler = "handler.lambda_handler" role = aws_iam_role.lambda_role.arn runtime = local.python_version layers = [aws_lambda_layer_version.lambda_layer.arn] lifecycle { create_before_destroy = true } }`  Copy Time to test To test if everything works, we have to add Terraform provider. In our case the provider is AWS. Let’s add a file **provider.tf** in terraform directory.  `# terraform/provider.tf provider "aws" { region = "eu-west-1" profile = "aws-profile-name-from-aws-config-file-at-your-machine" }`  Copy The project will look like  `. ├── lambda │ └── handler.py ├── Pipfile ├── Pipfile.lock └── terraform ├── build_layer.sh ├── lambda_layer.tf ├── lambda.tf ├── provider.tf └── requirements_creator.py`  Copy Let’s build infrastructure   `# Activate pipenv virtual environment pipenv shell # Go to terraform directory cd terraform # Initialize terraform terraform init # Check terraform infrastructure component to deploy terraform plan # And deploy with terraform apply # If you want to destroy all terraform destroy`  Copy ![Mahabubur Rahaman Melon](https://www.craftsmensoftware.com/hs-fs/hubfs/Mahabubur%20Rahaman%20Melon.png?width=120&height=120&name=Mahabubur%20Rahaman%20Melon.png) Author: **[Mahabubur Rahaman Melon](https://www.craftsmensoftware.com/blog/author/mahabubur-rahaman-melon)** 30/04/2024 Software Development Engineer ![Hire AI-Native Engineers](https://www.craftsmensoftware.com/hubfs/Z63_3132.webp) ## Hire AI-Native Engineers [![Book A Strategy Call](https://www.craftsmensoftware.com/hubfs/Book%20A%20Strategy%20Call.svg)](https://www.craftsmensoftware.com/contact-us) ### **About Craftsmen** [The Team](https://www.craftsmensoftware.com/about-us) [Our Partnership Model](https://www.craftsmensoftware.com/how-we-work) [Career](https://careers.craftsmensoftware.com/) [FAQ](https://www.craftsmensoftware.com/frequently-asked-questions-faq) [Privacy Policy](https://www.craftsmensoftware.com/privacy-policy-best-software-development-company-in-bangladesh)  [Craftsmen LinkedIn Page](https://linkedin.com/company/craftsmensoftware)   [Craftsmen Facebook Page](https://www.facebook.com/craftsmenltd/)   [Craftsmen Youtube Page](https://www.youtube.com/@craftsmensoftware) ### **Headquarter** **Craftsmen UK** One Sovereign Quay, Bridge End, Leeds, West Yorkshire LS1 4AG United Kingdom  **Operational Hub Craftsmen Bangladesh** Plot # 316, Lane # 4, DOHS Baridhara, Dhaka - 1206, Bangladesh ### **Sales & Partnership Offices** **Craftsmen Norway** Kong Oscars gate 66, 68, 5017 Bergen, Norway **Craftsmen France** 6 Avenue Pierre Grenier, 92100 Boulogne-Billancourt, Paris, France ### **A Team You Can Trust** #### [![ISO Certificates New](https://www.craftsmensoftware.com/hs-fs/hubfs/ISO%20Certificates%20New.png?width=300&height=96&name=ISO%20Certificates%20New.png)](https://www.craftsmensoftware.com/trust-centre) [![glassdoor (1)](https://www.craftsmensoftware.com/hs-fs/hubfs/glassdoor%20(1).png?width=110&height=79&name=glassdoor%20(1).png)](https://www.glassdoor.com/Overview/Working-at-Craftsmen-EI_IE3004266.11,20.htm) [![goodfirms.co](https://www.craftsmensoftware.com/hs-fs/hubfs/goodfirms.co.webp?width=110&height=95&name=goodfirms.co.webp)](https://www.craftsmensoftware.com/blog/craftsmen-thrives-at-goodfirms-by-offering-scalable-robust-software-solutions-developed-with-quality-by-maestros)    © 2026 Craftsmen. All rights reserved. ```json { "@context" : "https://schema.org", "@type" : "BlogPosting", "author" : { "@type" : "Person", "name" : "Mahabubur Rahaman Melon", "url" : "https://www.craftsmensoftware.com/blog/author/mahabubur-rahaman-melon" }, "dateModified" : "2026-04-30T11:42:58.240Z", "datePublished" : "2024-04-29T18:00:00.000Z", "headline" : "Automate python libraries deployment at AWS Lambda layer from Pipfile with Terraform - Best Software Development Team Extension Partner for Nordics", "image" : [ "https://www.craftsmensoftware.com/hubfs/Imported_Blog_Media/Automate_Python_Libraries_Deployment_At_AWS_Lambda_Layer.webp" ], "mainEntityOfPage" : { "@id" : "https://www.craftsmensoftware.com/blog/automate-python-libraries-deployment-at-aws-lambda-layer-from-pipfile-with-terraform", "@type" : "WebPage" }, "publisher" : { "@type" : "Organization", "logo" : { "@type" : "ImageObject", "url" : "https://www.craftsmensoftware.com/hubfs/Craftsmen%20Logo_Black.png" }, "name" : "Craftsmen Ltd." } } ``` ```json { "@context" : "https://schema.org", "@type" : "SoftwareCompany", "address" : [ { "@type" : "PostalAddress", "addressCountry" : "Norway", "addressLocality" : "Oslo" }, { "@type" : "PostalAddress", "addressCountry" : "United Kingdom", "addressLocality" : "London" }, { "@type" : "PostalAddress", "addressCountry" : "France", "addressLocality" : "Paris" }, { "@type" : "PostalAddress", "addressCountry" : "United States", "addressLocality" : "New York" }, { "@type" : "PostalAddress", "addressCountry" : "Bangladesh", "addressLocality" : "Dhaka" } ], "areaServed" : [ { "@type" : "Country", "name" : "Norway" }, { "@type" : "Country", "name" : "Sweden" }, { "@type" : "Country", "name" : "United Kingdom" }, { "@type" : "Country", "name" : "France" }, { "@type" : "Country", "name" : "United States" }, { "@type" : "Country", "name" : "Bangladesh" } ], "description" : "Trusted AI-native engineering partner providing software team extension, IT staff augmentation, and dedicated developer teams globally.", "logo" : "https://www.craftsmensoftware.com/hubfs/logo.png", "name" : "Craftsmen Software", "url" : "https://www.craftsmensoftware.com" } ```