Hoe Postman scripts uit te voeren via Gitlab Ci

Newman & Postman: API Testing in CI/CD Pipelines

Newman is de command-line runner voor Postman collecties, waarmee je API tests kunt automatiseren in je development workflow. Deze handleiding toont je hoe je effectief API testing kunt implementeren in je CI/CD pipeline.

Wat is Newman?

Newman is een krachtige tool die:

Postman collecties uitvoert vanaf de command line
Naadloos integreert met CI/CD pipelines
Uitgebreide rapportage biedt in verschillende formaten
Automatisch API testing mogelijk maakt

Voordelen van API Testing Automatisering

Vroege Bug Detectie: Vind API problemen voordat ze production bereiken Consistente Testing: Gestandaardiseerde tests op elke code wijziging Snellere Feedback: Automatische test resultaten bij elke commit Betrouwbaarheid: Zorg dat API’s voldoen aan specificaties voor deployment

Stap 1: API Tests Ontwikkelen in Postman

Een Test Collectie Maken

Nieuwe Collectie: Open Postman en maak een nieuwe collectie
API Request Toevoegen: Configureer je API endpoint met de juiste parameters
Environment Variables: Stel variabelen in voor verschillende omgevingen

Test Scripts Schrijven

In het “Tests” tabblad van je request:

// Basis status code test
pm.test("API returns successful status", function () {
    pm.response.to.have.status(200);
});

// Response tijd validatie
pm.test("Response time is acceptable", function () {
    pm.expect(pm.response.responseTime).to.be.below(1000);
});

// JSON schema validatie
pm.test("Response has correct structure", function () {
    const responseJson = pm.response.json();
    pm.expect(responseJson).to.have.property('result');
    pm.expect(responseJson.result).to.be.a('number');
});

// Business logic test
pm.test("Bonus calculation is correct", function () {
    const responseJson = pm.response.json();
    const salary = pm.environment.get("salary");
    const bonus = pm.environment.get("bonus_percentage");
    const expectedResult = salary * (bonus / 100);
    
    pm.expect(responseJson.result).to.equal(expectedResult);
});

// Environment variabelen instellen voor volgende tests
pm.test("Set calculated bonus for next test", function () {
    const responseJson = pm.response.json();
    pm.environment.set("calculated_bonus", responseJson.result);
});

Environment Configuration

Maak environment variabelen voor verschillende test scenario’s:

{
  "salary": 50000,
  "bonus_percentage": 10,
  "api_key": "{{API_KEY}}",
  "base_url": "{{BASE_URL}}"
}

Stap 2: Collectie Exporteren

Collectie Exporteren:
- Klik op “…” naast je collectie naam
- Selecteer “Export” → kies “Collection v2.1”
- Sla op als api_tests_collection.json
Environment Exporteren:
- Ga naar Environments
- Klik op “…” naast je environment
- Export als test_environment.json

Stap 3: Newman Project Setup

Dependencies Installeren

# Core Newman package
npm install newman --save-dev

# Enhanced HTML reporter
npm install newman-reporter-htmlextra --save-dev

# Voor JUnit XML output (CI integratie)
npm install newman-reporter-junitfull --save-dev

Newman Runner Script

Maak run-api-tests.js:

const newman = require('newman');
const path = require('path');

const runNewman = (environment = 'test') => {
    return new Promise((resolve, reject) => {
        newman.run({
            collection: path.join(__dirname, 'collections', 'api_tests_collection.json'),
            environment: path.join(__dirname, 'environments', `${environment}_environment.json`),
            
            // Global variables (overrides environment)
            globals: {
                api_key: process.env.API_KEY || 'default_test_key',
                base_url: process.env.BASE_URL || 'https://api.test.com'
            },
            
            // Reporting configuratie
            reporters: ['cli', 'htmlextra', 'junitfull'],
            reporter: {
                htmlextra: {
                    export: `./reports/api-test-report-${Date.now()}.html`,
                    template: './templates/custom-template.hbs', // Optioneel custom template
                    logs: true,
                    darkTheme: false,
                    testPaging: true,
                    browserTitle: "API Test Results",
                    title: "API Testing Report",
                    titleSize: 4,
                    omitHeaders: false,
                    skipHeaders: "Authorization,X-API-Key", // Gevoelige headers verbergen
                    hideRequestBody: ["password", "token"], // Gevoelige data verbergen
                    hideResponseBody: ["password", "token"]
                },
                junitfull: {
                    export: `./reports/junit-report-${Date.now()}.xml`
                }
            },
            
            // Newman configuratie
            bail: false, // Continue bij gefaalde tests
            suppressExitCode: false, // Exit met error code bij gefaalde tests
            verbose: true,
            color: 'on',
            
            // Request configuratie
            timeout: 30000, // 30 seconden timeout
            insecure: false, // SSL verificatie
            ignoreRedirects: false
            
        }, function (err, summary) {
            if (err) {
                console.error('Newman run failed:', err);
                reject(err);
                return;
            }
            
            // Test resultaten loggen
            console.log('\n📊 Test Summary:');
            console.log(`Total requests: ${summary.run.stats.requests.total}`);
            console.log(`Failed requests: ${summary.run.stats.requests.failed}`);
            console.log(`Total tests: ${summary.run.stats.tests.total}`);
            console.log(`Failed tests: ${summary.run.stats.tests.failed}`);
            console.log(`Total assertions: ${summary.run.stats.assertions.total}`);
            console.log(`Failed assertions: ${summary.run.stats.assertions.failed}`);
            
            if (summary.run.stats.tests.failed > 0 || summary.run.stats.assertions.failed > 0) {
                console.error('❌ Some tests failed!');
                reject(new Error('Test failures detected'));
            } else {
                console.log('✅ All tests passed!');
                resolve(summary);
            }
        });
    });
};

// Export voor gebruik in andere scripts
module.exports = { runNewman };

// Direct uitvoeren als main script
if (require.main === module) {
    const environment = process.argv[2] || 'test';
    
    console.log(`🚀 Starting API tests for ${environment} environment...`);
    
    runNewman(environment)
        .then(() => {
            console.log('🎉 Test run completed successfully!');
            process.exit(0);
        })
        .catch((error) => {
            console.error('💥 Test run failed:', error.message);
            process.exit(1);
        });
}

Package.json Scripts

{
  "scripts": {
    "test:api": "node run-api-tests.js",
    "test:api:dev": "node run-api-tests.js dev",
    "test:api:staging": "node run-api-tests.js staging",
    "test:api:prod": "node run-api-tests.js prod"
  }
}

Stap 4: CI/CD Pipeline Integratie

GitLab CI/CD Configuration

.gitlab-ci.yml:

# Docker image met Node.js
image: node:18-alpine

# Pipeline stages
stages:
  - install
  - lint
  - test
  - api-test
  - deploy

# Cache dependencies tussen jobs
cache:
  paths:
    - node_modules/
    - .npm/

# Variables
variables:
  NODE_ENV: test
  API_KEY: $TEST_API_KEY
  BASE_URL: $TEST_BASE_URL

# Dependencies installeren
install_dependencies:
  stage: install
  script:
    - npm ci --cache .npm --prefer-offline
  artifacts:
    paths:
      - node_modules/
    expire_in: 1 hour

# API Testing job
api_tests:
  stage: api-test
  dependencies:
    - install_dependencies
  before_script:
    - mkdir -p reports
    - echo "🔧 Setting up API testing environment..."
  script:
    - echo "🧪 Running API tests..."
    - npm run test:api
  after_script:
    - echo "📋 Test execution completed"
  artifacts:
    when: always
    paths:
      - reports/
    reports:
      junit: reports/junit-report-*.xml
    expire_in: 1 week
  coverage: '/Tests\s+:\s+(\d+\.\d+)%/'
  
# Production API tests (manual trigger)
api_tests_production:
  stage: api-test
  script:
    - npm run test:api:prod
  when: manual
  only:
    - main
  environment:
    name: production
    url: $PROD_BASE_URL

GitHub Actions Configuration

.github/workflows/api-tests.yml:

name: API Tests

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]
  schedule:
    - cron: '0 6 * * *'  # Daily at 6 AM UTC

jobs:
  api-tests:
    runs-on: ubuntu-latest
    
    strategy:
      matrix:
        node-version: [16, 18, 20]
        environment: [test, staging]
    
    steps:
    - name: Checkout code
      uses: actions/checkout@v4
    
    - name: Setup Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v4
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'npm'
    
    - name: Install dependencies
      run: npm ci
    
    - name: Create reports directory
      run: mkdir -p reports
    
    - name: Run API tests
      env:
        API_KEY: ${{ secrets.TEST_API_KEY }}
        BASE_URL: ${{ vars.TEST_BASE_URL }}
        NODE_ENV: ${{ matrix.environment }}
      run: npm run test:api:${{ matrix.environment }}
    
    - name: Upload test results
      uses: actions/upload-artifact@v4
      if: always()
      with:
        name: api-test-results-${{ matrix.node-version }}-${{ matrix.environment }}
        path: reports/
        retention-days: 30
    
    - name: Publish test results
      uses: dorny/test-reporter@v1
      if: always()
      with:
        name: API Tests Results
        path: reports/junit-report-*.xml
        reporter: java-junit

Stap 5: Geavanceerde Features

Dynamic Data Testing

// Pre-request script voor dynamic data
pm.sendRequest({
    url: pm.environment.get("base_url") + "/setup-test-data",
    method: 'POST',
    header: {
        'Authorization': 'Bearer ' + pm.environment.get("auth_token")
    },
    body: {
        mode: 'raw',
        raw: JSON.stringify({
            scenario: "bonus_calculation_test"
        })
    }
}, function (err, response) {
    if (!err) {
        const testData = response.json();
        pm.environment.set("test_employee_id", testData.employee_id);
        pm.environment.set("test_salary", testData.salary);
    }
});

Custom Newman Reporters

// custom-reporter.js
class CustomReporter {
    constructor(newman, reporterOptions) {
        newman.on('start', (err, args) => {
            console.log('🚀 API Test Suite Started');
        });
        
        newman.on('request', (err, args) => {
            if (err) {
                console.log(`❌ Request failed: ${args.item.name}`);
            } else {
                console.log(`✅ Request succeeded: ${args.item.name} (${args.response.responseTime}ms)`);
            }
        });
        
        newman.on('done', (err, summary) => {
            // Verstuur resultaten naar monitoring systeem
            this.sendToMonitoring(summary);
        });
    }
    
    sendToMonitoring(summary) {
        // Integratie met Slack, Teams, of monitoring tools
        const webhook = process.env.SLACK_WEBHOOK;
        if (webhook) {
            // Verstuur notificatie
        }
    }
}

module.exports = CustomReporter;

Best Practices

1. Test Data Management

// Gebruik van CSV data files
newman.run({
    collection: 'collection.json',
    iterationData: 'test-data.csv', // External data source
    iterationCount: 10
});

2. Environment Management

Development: Lokale API endpoints
Testing: Geïsoleerde test database
Staging: Production-like environment
Production: Alleen kritieke smoke tests

3. Security & Secrets

# GitLab CI/CD variables (masked)
variables:
  API_KEY: $MASKED_API_KEY
  DATABASE_URL: $PROTECTED_DB_URL

4. Error Handling

// Robust error handling in tests
pm.test("Handle API errors gracefully", function () {
    if (pm.response.code >= 400) {
        const error = pm.response.json();
        console.log(`API Error: ${error.message}`);
        
        // Log voor debugging
        pm.globals.set("last_error", JSON.stringify(error));
    }
    
    pm.expect(pm.response.code).to.be.oneOf([200, 201, 204]);
});

Monitoring & Alerting

Slack Integratie

// slack-notifier.js
const sendSlackNotification = (summary) => {
    const webhook = process.env.SLACK_WEBHOOK;
    const color = summary.run.stats.tests.failed > 0 ? 'danger' : 'good';
    
    const payload = {
        channel: '#api-testing',
        username: 'Newman API Tests',
        attachments: [{
            color: color,
            title: 'API Test Results',
            fields: [
                {
                    title: 'Total Tests',
                    value: summary.run.stats.tests.total,
                    short: true
                },
                {
                    title: 'Failed Tests',
                    value: summary.run.stats.tests.failed,
                    short: true
                }
            ]
        }]
    };
    
    // Verstuur naar Slack
    require('axios').post(webhook, payload);
};

Troubleshooting

Veelvoorkomende Problemen

SSL Certificate Issues:

newman.run({
    insecure: true, // Alleen voor development
    ignoreRedirects: false
});

Memory Issues bij Large Collections:

node --max-old-space-size=4096 run-api-tests.js

Rate Limiting:

newman.run({
    delayRequest: 1000, // 1 seconde tussen requests
    timeoutRequest: 30000
});

Conclusie

Newman en Postman bieden een krachtige combinatie voor API testing automatisering. Door deze tools te integreren in je CI/CD pipeline zorg je voor:

Consistente API kwaliteit door geautomatiseerde tests
Vroege bug detectie in de development cyclus
Betrouwbare deployments met pre-deployment validatie
Uitgebreide rapportage voor stakeholders

Volgende Stappen:

Start met eenvoudige API tests in Postman
Implementeer Newman in je lokale development workflow
Integreer geleidelijk in je CI/CD pipeline
Voeg monitoring en alerting toe voor production

Door Ralph van der Horst | GitLab Project

Nuttige Resources:

learnautomatedtesting.com