brefphp/benchmarks

Benchmarks for Bref running on AWS Lambda.

Scenarios

• PHP function: a simple PHP function, see function/index.php
• HTTP application: a simple PHP web page that returns Hello world, see fpm/index.php

Median (warm) execution time

Average execution time for a lambda that doesn't do anything.

Number of samples: 900

Bref 2.x (PHP 8.3)

Bref 2.x ARM (PHP 8.3)

For comparison on a 512M Digital Ocean droplet we get 1ms for "HTTP application" and 6ms for Symfony.

CPU performance

The more RAM, the more CPU power is allocated to the lambda. This is clearly visible when running PHP's official bench.php script.

For comparison bench.php runs in 1.3s on a 512M Digital Ocean droplet, in 0.8s on a 2.8Ghz i7 and in 0.6s on a 3.2Ghz i5.

Cold starts

Number of samples: 20

High level learnings:

• Don't use less than 512M of memory
• Cold starts are not faster on ARM, they are same or slightly slower
• Containers have the fastest cold starts
• When using containers, you need to warm up the first requests following a deployment

Comparison (1024M):

Bref 2.x containers (PHP 8.3)

Note: the first cold starts are much slower (see https://aaronstuyvenberg.com/posts/containers-on-lambda), e.g. 5s, because the container cache is warming up. In production warm up Lambda after deploying.

Function duration:

Total latency (measured from API Gateway or X-Ray):

Bref 2.x (PHP 8.3)

Function duration:

Total latency (measured from API Gateway or X-Ray):

Bref 2.x ARM (PHP 8.3)

Function duration:

Total latency (measured from API Gateway or X-Ray):

Measuring cold starts in CloudWatch Logs Insights:

filter @type = “REPORT” and @initDuration
| stats
 count(@type) as count,
 min(@billedDuration) as min,
 avg(@billedDuration) as avg,
 pct(@billedDuration, 50) as p50,
 max(@billedDuration) as max
by @log
| sort @log

Reproducing

You will need to install dependencies of Bref. Then:

• clone the repository
• make setup
• make deploy

Then run the make bench-* commands.