Leveraging on a serverless Pact Broker with PostgreSQL on AWS ECS fargate

Introduction

Introducing a Serverless Pact Broker with PostgreSQL on AWS ECS Fargate

Introduction

A partner recently asked me to demonstrate how contract testing, also known as Pact, works for one of our clients. Although I hadn’t used Pact in practice, my extensive experience in integration testing and service virtualization made me eager to explore. My goal was to establish a serverless Pact Broker on AWS ECS (Elastic Container Service) for both the Pact Broker and its PostgreSQL database. This guide outlines how to set up a Pact Broker and PostgreSQL on ECS using AWS Cloud Development Kit (CDK) in Python, focusing on cost-efficiency and scalability.

Prerequisites

• AWS account with proper access
• AWS CLI and AWS CDK installed
• Docker installed for local container image management
• Basic understanding of Docker, AWS ECS, and networking

Architecture Overview

The solution comprises:

• Amazon ECS: Hosts both the Pact Broker and PostgreSQL in containers.
• Amazon S3: Stores Pact files for versioning and sharing.
• Future AWS Lambda Integration: Automates pact verification processes.
• Amazon CloudWatch: Provides logging and monitoring.

Cost Considerations

• Opt for ECS Fargate Spot Instances for cost savings in non-production environments.
• Regularly review ECS task usage and adjust task sizing as necessary to avoid over-provisioning.
• Utilize S3 lifecycle policies to minimize storage costs.
• Explore the AWS Free Tier for eligible services to reduce initial expenses.

Implementation Steps

1. ECS Cluster and Networking

Set up a VPC and an ECS cluster to provide a networked environment for the containerized services:

t2. Containerized PostgreSQL Database

Deploy PostgreSQL as a container within ECS. Define a Dockerfile for PostgreSQL or use an existing image from Docker Hub. Ensure persistent storage through ECS volume management

3. Pact Broker Deployment

Containerize the Pact Broker and deploy it on ECS. Ensure it’s configured to communicate with the PostgreSQL container:

After implementation Verifying and Deploying Pacts

First, verify a pact and then deploy it to the broker. After verification, your setup should look something like this:

import { expect } from 'chai';
import path from 'path';
import { Pact, Matchers } from '@pact-foundation/pact';
import { getMeBSN, getMeBSNs } from './index.mjs';

const { like } = Matchers;

describe('The BSN API', () => {
 let url = 'localhost';
 const port = 8992;

 const provider = new Pact({
   port: port,
   log: path.resolve(process.cwd(), 'logs', 'mockserver-integration.log'),
   dir: path.resolve(process.cwd(), 'pacts'),
   spec: 2,
   consumer: 'RWS-consumer',
   provider: 'BRP-provider',
 //  logLevel: 'trace', // assuming you want to set log level to 'trace'
 });

 const EXPECTED_BODY = [
   {
     bsn: 123456789,
   },
   {
     bsn: 987654321,
   },
 ];

 // Setup the provider
 before(async () => {
   await provider.setup();
 });

 // Write Pact when all tests done
 after(async () => {
   await provider.finalize();
 });

 // verify with Pact, and reset expectations
 afterEach(async () => {
   await provider.verify();
 });

 describe('get /bsns', () => {
   before(async () => {
     const interaction = {
       state: 'i have a list of BSN',
       uponReceiving: 'a request for all BSNs',
       withRequest: {
         method: 'GET',
         path: '/bsns',
         headers: {
           Accept: [
             'application/problem+json',
             'application/json',
             'text/plain',
             '*/*',
           ],
         },
       },
       willRespondWith: {
         status: 200,
         headers: {
           'Content-Type': 'application/json',
         },
         body: [
           { bsn: 123456789 },
           { bsn: 987654321 }
         ],
       },
     };
     await provider.addInteraction(interaction);
   });

   it('returns the correct response', async () => {
     const urlAndPort = {
       url: url,
       port: port,
     };
     const response = await getMeBSNs(urlAndPort);
     console.log(response)
     expect(response).to.eql(EXPECTED_BODY);
   });
 });
});

After the deploy the pact is visible in the pactbroker stored in aws postgress ecs

To demonstrate that when you creating a provider and to see if it will fail on verify or not I created two apis in api gateway and lambda and change the structure of the second api a bit

The code for verifying pact as a provider is this

import axios from 'axios';
import fs from 'fs/promises'; // Use fs.promises for async/await support
import { Verifier } from '@pact-foundation/pact';

const pactFileUrl = 'http://pactst-pactb-oesxbucvqxww-103295103.eu-west-2.elb.amazonaws.com/pacts/provider/BRP-provider/consumer/RWS-consumer/latest';
const localFilePath = './local-pact-file.json'; // Path where you want to save the pact file

// Basic Authentication credentials
const username = 'admin';
const password = 'password';
const auth = Buffer.from(`${username}:${password}`).toString('base64');

async function downloadPactFile() {
 try {
   const response = await axios.get(pactFileUrl, {
     headers: {
       'Authorization': `Basic ${auth}`
     },
     responseType: 'json' // Assuming the response is JSON
   });

   // Save the file locally
   await fs.writeFile(localFilePath, JSON.stringify(response.data));
   console.log(`Pact file downloaded and saved to ${localFilePath}`);
 } catch (error) {
   console.error('Failed to download the pact file:', error.message);
 }
}


async function verifyPacts() {
 // Path to the local pact file
 const localPactPath = './local-pact-file.json';

 const opts = {
   provider: 'BRP-provider',
   providerBaseUrl: 'https://sqsgrhcuji.execute-api.eu-west-2.amazonaws.com/prod',
   pactUrls: [localPactPath], // Use the local file path
   logLevel: 'DEBUG',
 };

 try {
   await new Verifier(opts).verifyProvider();
   console.log('Pact verification complete!');
 } catch (error) {
   console.error('Pact verification failed:', error.message);
   process.exit(1); // Exit
 }
}

async function run() {
 await downloadPactFile();
 await verifyPacts();
}

run();

When running my pipeline test you will see the third test failing

The Result

Running my pipeline tests shows the third test failing due to a broken pact. The logs clearly indicate the issue: a change from bsn to bsnnew. This demonstrates the MVP of creating a serverless demo by setting up dummy APIs in Lambda.

Future Enhancements

1. S3 for Pact Files: Configure an S3 bucket for pact file storage and management.
2. Lambda for Pact Verification: Plan to use AWS Lambda for automated pact verification triggered by S3 uploads.

Conclusion

Deploying the Pact Broker and PostgreSQL on AWS ECS provides a scalable, efficient, and cost-effective solution for contract testing in a microservices environment. Future enhancements, including AWS Lambda integration, promise to further automate and streamline the contract testing process.

For more detailed information and to access the repositories for this project, visit:

• https://gitlab.com/learnautomatedtesting/pactexample (API Examples Provider)
• https://gitlab.com/learnautomatedtesting/servicevirtualizationandpact/ (Example opensource Serverless stack AWS CDK Python v2) This setup showcases a practical implementation of Pact in AWS, emphasizing serverless architecture’s benefits for contract testing.