How to set up Laravel with Artisan Commands, Controllers, a Queue + workers and a Database on Docker

This article appeared first on https://www.pascallandau.com/ at Run Laravel 9 on Docker in 2022 [Tutorial Part 4]

---

In this part of my tutorial series on developing PHP on Docker we will install Laravel and make sure our setup works for Artisan Commands, a Redis Queue and Controllers for the front end requests.

All code samples are publicly available in my Docker PHP Tutorial repository on Github. You find the branch for this tutorial at part-4-3-run-laravel-9-docker-in-2022.

All published parts of the Docker PHP Tutorial are collected under a dedicated page at Docker PHP Tutorial. The previous part was PhpStorm, Docker and Xdebug 3 on PHP 8.1 in 2022 and the following one is Set up PHP QA tools and control them via make.

If you want to follow along, please subscribe to the RSS feed or via email to get automatic notifications when the next part comes out :)

Table of contents

• Introduction
• Install extensions
• Install Laravel
• Update the PHP POC
  • config
    • database connection
    • queue connection
  • Controllers
  • Commands
  • Jobs and workers
  • Tests
• Makefile updates
  • Clearing the queue
• Running the POC
• Wrapping up

Introduction

The goal of this tutorial is to run the PHP POC from part 4.1 using Laravel as a framework instead of "plain PHP". We'll use the newest version of Laravel (Laravel 9) that was released at the beginning of February 2022.

Install extensions

Before Laravel can be installed, we need to add the necessary extensions of the framework (and all its dependencies) to the php-base image:

# File: .docker/images/php/base/Dockerfile

# ...

RUN apk add --update --no-cache  \
        php-curl~=${TARGET_PHP_VERSION} \

Install Laravel

We'll start by creating a new Laravel project with composer

composer create-project --prefer-dist laravel/laravel /tmp/laravel "9.*" --no-install --no-scripts

The files are added to /tmp/laravel because composer projects cannot be created in non-empty folders , so we need to create the project in a temporary location first and move it afterwards.

Since I don't have PHP 8 installed on my laptop, I'll execute the command in the application docker container via

make execute-in-container DOCKER_SERVICE_NAME="application" COMMAND='composer create-project --prefer-dist laravel/laravel /tmp/laravel "9.*" --no-install --no-scripts'

and then move the files into the application directory via

rm -rf public/ tests/ composer.* phpunit.xml
make execute-in-container DOCKER_SERVICE_NAME="application" COMMAND="bash -c 'mv -n /tmp/laravel/{.*,*} .' && rm -f /tmp/laravel"
cp .env.example .env

Notes

• composer install is skipped via --no-install to avoid having to copy over the vendor/ folder (which is super slow on Docker Desktop)
• existing directories cannot be overwritten by mv thus I remove public/ and tests/ upfront (as well as the composer and phpunit config files)
• mv uses the -n flag so that existing files like our .editorconfig are not overwritten
• I need to use bash -c to run the command in the container because otherwise the * wildcard would have no effect in the container

To finalize the installation I need to install the composer dependencies and execute the create-project scripts defined in composer.json:

make composer ARGS=install
make composer ARGS="run-script post-create-project-cmd"

Since our nginx configuration was already pointing to the public/ directory, we can immediately open http://127.0.0.1 in the browser and should see the frontpage of a fresh Laravel installation.

Update the PHP POC

config

We need to update the connection information for the database and the queue (previously configured via dependencies.php) in the .env file

database connection

DB_CONNECTION=mysql
DB_HOST=mysql
DB_PORT=3306
DB_DATABASE=application_db
DB_USERNAME=root
DB_PASSWORD=secret_mysql_root_password

queue connection

QUEUE_CONNECTION=redis

REDIS_HOST=redis
REDIS_PASSWORD=secret_redis_password

Controllers

The functionality of the previous public/index.php file now lives in the HomeController at app/Http/Controllers/HomeController.php

class HomeController extends Controller
{
    use DispatchesJobs;

    public function __invoke(Request $request, QueueManager $queueManager, DatabaseManager $databaseManager): View
    {
        $jobId = $request->input("dispatch") ?? null;
        if ($jobId !== null) {
            $job = new InsertInDbJob($jobId);
            $this->dispatch($job);

            return $this->getView("Adding item '$jobId' to queue");
        }

        if ($request->has("queue")) {

            /**
             * @var RedisQueue $redisQueue
             */
            $redisQueue = $queueManager->connection();
            $redis =  $redisQueue->getRedis()->connection();
            $queueItems = $redis->lRange("queues:default", 0, 99999);

            $content = "Items in queue\n".var_export($queueItems, true);

            return $this->getView($content);
        }

        if ($request->has("db")) {
            $items = $databaseManager->select($databaseManager->raw("SELECT * FROM jobs"));

            $content = "Items in db\n".var_export($items, true);

            return $this->getView($content);
        }
        $content = <<<HTML
            <ul>
                <li><a href="?dispatch=foo">Dispatch job 'foo' to the queue.</a></li>
                <li><a href="?queue">Show the queue.</a></li>
                <li><a href="?db">Show the DB.</a></li>
            </ul>
            HTML;

        return $this->getView($content);
    }

    private function getView(string $content): View
    {
        return view('home')->with(["content" => $content]);
    }
}

Its content is displayed via the home view located at resources/views/home.blade.php:

<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
    </head>
    <body>
    {!! $content !!}
    </body>
</html>

The controller is added as a route in routes/web.php:

Route::get('/', \App\Http\Controllers\HomeController::class)->name("home");

Commands

We will replace the setup.php script with a SetupDbCommand that is located at app/Commands/SetupDbCommand.php

class SetupDbCommand extends Command
{
    /**
     * @var string
     */
    protected $name = "app:setup-db";

    /**
     * @var string
     */
    protected $description = "Run the application database setup";

    protected function getOptions(): array
    {
        return [
            [
                "drop",
                null,
                InputOption::VALUE_NONE,
                "If given, the existing database tables are dropped and recreated.",
            ],
        ];
    }

    public function handle()
    {
        $drop = $this->option("drop");
        if ($drop) {
            $this->info("Dropping all database tables...");

            $this->call(WipeCommand::class);

            $this->info("Done.");
        }

        $this->info("Running database migrations...");

        $this->call(MigrateCommand::class);

        $this->info("Done.");
    }
}

Register it the AppServiceProvider in app/Providers/AppServiceProvider.php

    public function register()
    {
        $this->commands([
            \App\Commands\SetupDbCommand::class
        ]);
    }

and update the setup-db target in .make/01-00-application-setup.mk to run the artisan Command

.PHONY: setup-db
setup-db: ## Setup the DB tables
    $(EXECUTE_IN_APPLICATION_CONTAINER) php artisan app:setup-db $(ARGS);

We will also create a migration for the jobs table in database/migrations/2022_02_10_000000_create_jobs_table.php:

return new class extends Migration
{
    public function up(): void
    {
        Schema::create('jobs', function (Blueprint $table) {
            $table->id();
            $table->string('value');
        });
    }
};

Jobs and workers

We will replace the worker.php script with InsertInDbJob located at app/Jobs/InsertInDbJob.php

class InsertInDbJob implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable;

    public function __construct(
        public readonly string $jobId
    ) {
    }

    public function handle(DatabaseManager $databaseManager)
    {
        $databaseManager->insert("INSERT INTO `jobs`(value) VALUES(?)", [$this->jobId]);
    }
}

though this will "only" handle the insertion part. For the worker itself we will use the native \Illuminate\Queue\Console\WorkCommand via

php artisan queue:work

We need to adjust the .docker/images/php/worker/Dockerfile and change

ARG PHP_WORKER_COMMAND="php $APP_CODE_PATH/worker.php"

to

ARG PHP_WORKER_COMMAND="php $APP_CODE_PATH/artisan queue:work"

Since this change takes place directly in the Dockerfile, we must now rebuild the image

$ make docker-build-image DOCKER_SERVICE_NAME=php-worker

and restart it

$ make docker-up

Tests

I'd also like to take this opportunity to add a Feature test for the HomeController at tests/Feature/App/Http/Controllers/HomeControllerTest.php:

class HomeControllerTest extends TestCase
{
    public function setUp(): void
    {
        parent::setUp();

        $this->setupDatabase();
        $this->setupQueue();
    }

    /**
     * @dataProvider __invoke_dataProvider
     */
    public function test___invoke(array $params, string $expected): void
    {
        $urlGenerator = $this->getDependency(UrlGenerator::class);

        $url = $urlGenerator->route("home", $params);

        $response = $this->get($url);

        $response
            ->assertStatus(200)
            ->assertSee($expected, false)
        ;
    }

    public function __invoke_dataProvider(): array
    {
        return [
            "default"           => [
                "params"   => [],
                "expected" => <<<TEXT
                        <li><a href="?dispatch=foo">Dispatch job 'foo' to the queue.</a></li>
                        <li><a href="?queue">Show the queue.</a></li>
                        <li><a href="?db">Show the DB.</a></li>
                    TEXT
                ,
            ],
            "database is empty" => [
                "params"   => ["db"],
                "expected" => <<<TEXT
                        Items in db
                    array (
                    )
                    TEXT
                ,
            ],
            "queue is empty"    => [
                "params"   => ["queue"],
                "expected" => <<<TEXT
                        Items in queue
                    array (
                    )
                    TEXT
                ,
            ],
        ];
    }

    public function test_shows_existing_items_in_database(): void
    {
        $databaseManager = $this->getDependency(DatabaseManager::class);

        $databaseManager->insert("INSERT INTO `jobs` (id, value) VALUES(1, 'foo');");

        $urlGenerator = $this->getDependency(UrlGenerator::class);

        $params = ["db"];
        $url    = $urlGenerator->route("home", $params);

        $response = $this->get($url);

        $expected = <<<TEXT
                Items in db
            array (
              0 => 
              (object) array(
                 'id' => 1,
                 'value' => 'foo',
              ),
            )
            TEXT;

        $response
            ->assertStatus(200)
            ->assertSee($expected, false)
        ;
    }

    public function test_shows_existing_items_in_queue(): void
    {
        $queueManager = $this->getDependency(QueueManager::class);

        $job = new InsertInDbJob("foo");
        $queueManager->push($job);

        $urlGenerator = $this->getDependency(UrlGenerator::class);

        $params = ["queue"];
        $url    = $urlGenerator->route("home", $params);

        $response = $this->get($url);

        $expectedJobsCount = <<<TEXT
                Items in queue
            array (
              0 => '{
            TEXT;

        $expected = <<<TEXT
            \\\\"jobId\\\\";s:3:\\\\"foo\\\\";
            TEXT;

        $response
            ->assertStatus(200)
            ->assertSee($expectedJobsCount, false)
            ->assertSee($expected, false)
        ;
    }
}

The test checks the database as well as the queue and uses the helper methods $this->setupDatabase() and $this->setupQueue() that I defined in the base test case at tests/TestCase.php as follows

   /**
     * @template T
     * @param class-string<T> $className
     * @return T
     */
    protected function getDependency(string $className)
    {
        return $this->app->get($className);
    }

    protected function setupDatabase(): void
    {
        $databaseManager = $this->getDependency(DatabaseManager::class);

        $actualConnection  = $databaseManager->getDefaultConnection();
        $testingConnection = "testing";
        if ($actualConnection !== $testingConnection) {
            throw new RuntimeException("Database tests are only allowed to run on default connection '$testingConnection'. The current default connection is '$actualConnection'.");
        }

        $this->ensureDatabaseExists($databaseManager);

        $this->artisan(SetupDbCommand::class, ["--drop" => true]);
    }

    protected function setupQueue(): void
    {
        $queueManager = $this->getDependency(QueueManager::class);

        $actualDriver  = $queueManager->getDefaultDriver();
        $testingDriver = "testing";
        if ($actualDriver !== $testingDriver) {
            throw new RuntimeException("Queue tests are only allowed to run on default driver '$testingDriver'. The current default driver is '$actualDriver'.");
        }

        $this->artisan(ClearCommand::class);
    }

    protected function ensureDatabaseExists(DatabaseManager $databaseManager): void
    {
        $connection = $databaseManager->connection();

        try {
            $connection->getPdo();
        } catch (PDOException $e) {
            // e.g. SQLSTATE[HY000] [1049] Unknown database 'testing'
            if ($e->getCode() !== 1049) {
                throw $e;
            }
            $config             = $connection->getConfig();
            $config["database"] = "";

            $connector = new MySqlConnector();
            $pdo       = $connector->connect($config);
            $database  = $connection->getDatabaseName();
            $pdo->exec("CREATE DATABASE IF NOT EXISTS `{$database}`;");
        }
    }

The methods ensure that the tests are only executed if the proper database connection and queue driver is configured. This is done through environment variables and I like using a dedicated .env file located at .env.testing for all testing ENV values instead of defining them in the phpunit.xml config file via <env> elements:

# File: .env.testing

DB_CONNECTION=testing
DB_DATABASE=testing
QUEUE_CONNECTION=testing
REDIS_DB=1000

The corresponding connections have to be configured in the config files

# File: config/database.php

return [
// ...
    'connections' => [
// ...
        'testing' => [
            'driver' => 'mysql',
            'url' => env('DATABASE_URL'),
            'host' => env('DB_HOST'),
            'port' => env('DB_PORT', '3306'),
            'database' => env('DB_DATABASE', 'testing'),
            'username' => env('DB_USERNAME'),
            'password' => env('DB_PASSWORD', ''),
            'unix_socket' => env('DB_SOCKET', ''),
            'charset' => 'utf8mb4',
            'collation' => 'utf8mb4_unicode_ci',
            'prefix' => '',
            'prefix_indexes' => true,
            'strict' => true,
            'engine' => null,
            'options' => extension_loaded('pdo_mysql') ? array_filter([
                PDO::MYSQL_ATTR_SSL_CA => env('MYSQL_ATTR_SSL_CA'),
            ]) : [],
        ],
    ],
// ...
    'redis' => [
// ...
        'testing' => [
            'url' => env('REDIS_URL'),
            'host' => env('REDIS_HOST', '127.0.0.1'),
            'password' => env('REDIS_PASSWORD'),
            'port' => env('REDIS_PORT', '6379'),
            'database' => env('REDIS_DB', '1000'),
        ],
    ],
];

# File: config/queue.php

return [
// ...

    'connections' => [
// ...
        'testing' => [
            'driver' => 'redis',
            'connection' => 'testing', // => refers to the "database.redis.testing" config entry
            'queue' => env('REDIS_QUEUE', 'default'),
            'retry_after' => 90,
            'block_for' => null,
            'after_commit' => false,
        ],
    ],
];

The tests can be executed via make test

$ make test
ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose.yml -f ./.docker/docker-compose/docker-compose.local.yml exec -T --user application php-worker vendor/bin/phpunit -c phpunit.xml
PHPUnit 9.5.13 by Sebastian Bergmann and contributors.

.......                                                             7 / 7 (100%)

Time: 00:02.709, Memory: 28.00 MB

OK (7 tests, 13 assertions)

Makefile updates

Clearing the queue

For convenience while testing I added a make target to clear all items from the queue in .make/01-01-application-commands.mk

.PHONY: clear-queue
clear-queue: ## Clear the job queue
    $(EXECUTE_IN_APPLICATION_CONTAINER) php artisan queue:clear $(ARGS)

Running the POC

Since the POC only uses make targets and we basically just "refactored" them, there is no modification necessary to make the existing test.sh work:

$ bash test.sh


  Building the docker setup


//...


  Starting the docker setup


//...


  Clearing DB


ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose.yml -f ./.docker/docker-compose/docker-compose.local.yml exec -T --user application application php artisan app:setup-db --drop;
Dropping all database tables...
Dropped all tables successfully.
Done.
Running database migrations...
Migration table created successfully.
Migrating: 2014_10_12_000000_create_users_table
Migrated:  2014_10_12_000000_create_users_table (64.04ms)
Migrating: 2014_10_12_100000_create_password_resets_table
Migrated:  2014_10_12_100000_create_password_resets_table (50.06ms)
Migrating: 2019_08_19_000000_create_failed_jobs_table
Migrated:  2019_08_19_000000_create_failed_jobs_table (58.61ms)
Migrating: 2019_12_14_000001_create_personal_access_tokens_table
Migrated:  2019_12_14_000001_create_personal_access_tokens_table (94.03ms)
Migrating: 2022_02_10_000000_create_jobs_table
Migrated:  2022_02_10_000000_create_jobs_table (31.85ms)
Done.


  Stopping workers


ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose.yml -f ./.docker/docker-compose/docker-compose.local.yml exec -T --user application php-worker supervisorctl stop worker:*;
worker:worker_00: stopped
worker:worker_01: stopped
worker:worker_02: stopped
worker:worker_03: stopped


  Ensuring that queue and db are empty


<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
    </head>
    <body>
    Items in queue
array (
)
    </body>
</html>
<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
    </head>
    <body>
    Items in db
array (
)
    </body>
</html>


  Dispatching a job 'foo'


<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
    </head>
    <body>
    Adding item 'foo' to queue
    </body>
</html>


  Asserting the job 'foo' is on the queue


<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
    </head>
    <body>
    Items in queue
array (
  0 => '{"uuid":"7ea63590-2a86-4739-abf8-8a059d41bd60","displayName":"App\\\\Jobs\\\\InsertInDbJob","job":"Illuminate\\\\Queue\\\\CallQueuedHandler@call","maxTries":null,"maxExceptions":null,"failOnTimeout":false,"backoff":null,"timeout":null,"retryUntil":null,"data":{"commandName":"App\\\\Jobs\\\\InsertInDbJob","command":"O:22:\\"App\\\\Jobs\\\\InsertInDbJob\\":11:{s:5:\\"jobId\\";s:3:\\"foo\\";s:3:\\"job\\";N;s:10:\\"connection\\";N;s:5:\\"queue\\";N;s:15:\\"chainConnection\\";N;s:10:\\"chainQueue\\";N;s:19:\\"chainCatchCallbacks\\";N;s:5:\\"delay\\";N;s:11:\\"afterCommit\\";N;s:10:\\"middleware\\";a:0:{}s:7:\\"chained\\";a:0:{}}"},"id":"I3k5PNyGZc6Z5XWCC4gt0qtSdqUZ84FU","attempts":0}',
)
    </body>
</html>


  Starting the workers


ENV=local TAG=latest DOCKER_REGISTRY=docker.io DOCKER_NAMESPACE=dofroscra APP_USER_NAME=application APP_GROUP_NAME=application docker-compose -p dofroscra_local --env-file ./.docker/.env -f ./.docker/docker-compose/docker-compose.yml -f ./.docker/docker-compose/docker-compose.local.yml exec -T --user application php-worker supervisorctl start worker:*;
worker:worker_00: started
worker:worker_01: started
worker:worker_02: started
worker:worker_03: started


  Asserting the queue is now empty


<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
    </head>
    <body>
    Items in queue
array (
)
    </body>
</html>


  Asserting the db now contains the job 'foo'


<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
    </head>
    <body>
    Items in db
array (
  0 =>
  (object) array(
     'id' => 1,
     'value' => 'foo',
  ),
)
    </body>
</html>

Wrapping up

Congratulations, you made it! If some things are not completely clear by now, don't hesitate to leave a comment. Laravel 9 should now be up and running on the previously set up docker infrastructure.

In the next part of this tutorial, we will Set up PHP QA tools and control them via make.

Please subscribe to the RSS feed or via email to get automatic notifications when this next part comes out :)

;