Running Forme Job Queues Standalone

6 minutes read
forme framework • job queues • bernardphp • php • slim framework

Standalone Job Queue

I had a need for a simple job queue implementation for a project I’m working on, Promo by Recordlabel.io. The underlying (micro) framework is Slim 4 which is very much bare bones - a bit of underlying http glue really - with pretty much everything else provided by a patchwork of third party packages. For example I’m using Laravel’s Eloquent ORM, and a bunch of Cake and Symfony components amongst other packages from the likes of Spatie and PHP League. So I started searching for a framework agnostic solution, or at least one that could be used standalone.

Laravel Queues

I initially considered Laravel Queues, which is excellent and which I’ve obviously used extensively in Laravel projects. However, after a bit of research, while technically possible, it seemed a little bit laborious and hacky to get it working outside of a Laravel app.

Bernard, yo

I then stumbled across Bernard, which looked very promising. It aims to offer a unified queue processing api for a whole host of different backends, including Eloquent via its Laravel package. This seemed like a good shout, especially if the project eventually outgrows using a database queue, and I need to upgrade to something like Rabbit MQ without too much of a rewrite.

I did note that development on Bernard seems to have slowed right down - the docs are in a fairly parlous state, out of synch with the current release which is 3 years old - it’s not really clear which version they relate to. I got straight to work integrating it anyway, and got quite far down the line with it, however I quickly discovered that it is unfortunately fairly broken as of right now. After reaching a bit of a dead end with the current release, I tried switching to dev-master but even this had a bunch of ancient Symfony dependencies which were completely incompatible with my project. At this point I decided to cut my losses, while I could no doubt have powered through and solved all of the issues, this was starting to look like a lot more work than I had originally anticipated.

Forme Job Queue

I then had a brainwave - why not simply use Forme’s Job Queue? Forme is an MVC framework built on top of WordPress, but I realised the Job Queue component should be decoupled enough to use independently in any PHP project. It’s also very lightweight and simple, and I know it and its various limitations and quirks very well indeed, since I built it :-D

The Technical Bit

The first step was to install Forme:

composer require forme/framework:^v2

For reference the following is based on version 2.3.2 which is current as of this post’s publish date.

I then needed to copy over the job queue database migrations into my project. No problem at all and no conversion needed since I’m also using Phinx to handle those.

<?php
declare(strict_types=1);

use Phinx\Migration\AbstractMigration;

final class CreateFormeQueuedJobsTable extends AbstractMigration
{
    public function change(): void
    {
        $table = $this->table('forme_queued_jobs');
        $table->addColumn('class', 'string')
            ->addColumn('arguments', 'text')
            ->addColumn('started_at', 'datetime', ['null' => true])
            ->addColumn('completed_at', 'datetime', ['null' => true])
            ->addColumn('created_at', 'datetime')
            ->addColumn('updated_at', 'datetime')
            ->addColumn('scheduled_for', 'datetime')
            ->addColumn('frequency', 'string', ['null' => true])
            ->create();
    }
}

I was then able to run the migration locally to create the relevant db table.

// usually ./vendor/bin, I just prefer ./tools
./tools/phinx migrate

I then knew I would need to reimplement the Job Queue command since the Forme version relies on WP Cli, whereas my project uses Symfony Console. It’s a very simple command so this was straightforward enough.

<?php
declare(strict_types=1);

namespace App\Commands;

use Forme\Framework\Jobs\Queue;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

final class WorkQueue extends Command
{
    protected static $defaultName = 'work';
	  // todo: add some help text

    protected function execute(InputInterface $input, OutputInterface $output): int
    {
        $queue    = app()->get(Queue::class);
        $response = $queue->next();
        $output->writeln($response);

        return Command::SUCCESS;
    }
}

I then wired that into my exiting dev console app logic. I’m using my project’s app() container helper here to grab the instance.

$console->add(app()->get(WorkQueue::class));

All good, this command will eventually need to be triggered by a cron job running every minute on live deployment, but it means we are now able to run the next job in the queue manually.

I then added an empty job according to Forme’s docs. The actual logic to be implemented later,

<?php

declare(strict_types=1);

namespace App\Jobs;

use Forme\Framework\Jobs\JobInterface;
use Forme\Framework\Jobs\Queueable;

final class SendPromo implements JobInterface
{
    use Queueable;

    public function handle(array $args = []): ?string
    {
        // to do: implement email send

        return 'Send with id ' . $args['sendId'] . ' sent';
    }
}

And finally dispatched that from the relevant place in my app (the Insert Send controller, if you have to know, although that’s not really important for this discussion)

$this->sendPromo->dispatch(['sendId' => $send->id]);

Time to take it for a spin, but oh no, what’s this?

Error
Undefined constant "Forme\FORME_PRIVATE_ROOT"

Chasing back through the stack trace, we get to this line from vendor/forme/framework/src/Jobs/Queueable.php:

$queue = \Forme\getInstance(\Forme\Framework\Jobs\Queue::class);

Looks like there was a bit of coupling after all, although thankfully not with WordPress. Forme’s Queueable trait relies on Forme’s container helper function getInstance(), which in turn relies on Forme’s specific container being configured and available. No sweat, we’ll just recreate the trait using the current project’s app() helper instead.

<?php
declare(strict_types=1);

namespace App\Jobs;

use Forme\Framework\Jobs\Queue;

trait Queueable
{
    public function dispatch(array $args = []): void
    {
        $queue = app()->get(Queue::class);
        $queue->dispatch(['class' => $this::class, 'arguments' => $args]);
    }

    public function schedule(array $args = []): void
    {
        $queue         = app()->get(Queue::class);
        $args['class'] = $this::class;
        $queue->schedule($args);
    }

    public function start(array $args = []): void
    {
        $queue         = app()->get(Queue::class);
        $args['class'] = $this::class;
        $queue->start($args);
    }

    public function stop(): void
    {
        $queue = app()->get(Queue::class);
        $queue->stop(['class' => $this::class]);
    }
}

And then import that Queueable into our job instead of Forme’s one. All we have to do is remove the use statement since the new trait is in the same namespace as the job.

<?php

declare(strict_types=1);

namespace App\Jobs;

use Forme\Framework\Jobs\JobInterface;

final class SendPromo implements JobInterface
{
    use Queueable;

    public function handle(array $args = []): ?string
    {
        // to do: implement email send

        return 'Send with id ' . $args['sendId'] . ' sent';
    }
}

I triggered the job dispatch again and all present and correct in the DB this time.

Sequel Ace Screenshot

We can now run the work command from our console app to try running the job.

✦ ❯ ./wrangle work
Ran job 1: Send with id 98 sent

Bingo!

Next Up

Of course, there is plenty more to do, not least implementing the actual job logic, but I’m really pleased with how quick and easy this was to set up.

It’s also given me the idea to extract the Forme Job Queue logic into its own package, and make the PSR-11 container injectable so that we don’t rely on a helper function at all. Added to the backlog, so watch this space.

Update 2022-09-04

As of Forme 2.4.1, this is now even easier to integrate. The Queueable trait now supports different environments out of the box, including app() if it returns a PSR-11 instance, injecting a container into the job class constructor, or simply the queue itself.