Blog

One of the big challenges with updating Drupal Code Builder for Drupal 8 has been the sheer variety of code to be output. On earlier versions of Drupal, it was just about hooks, and all that needed to be done was to take the API documentation code and replace 'hook_' with the module name. There were info files too, and Drupal 7 added the placing of hooks into different .inc files, but compared to this, Drupal 8 has things like plugin annotations, fluent method calls for content entity baseFieldDefinitions(), FormAPI arrays, not to mention PHP class methods, and more.

But one of the things I enjoy about working on DCB is that I am free to experiment with different ideas, much more so than with work on core or even contrib. It is its own system, without any need to work with what a framework supplies, and it has no need to be extensible. So I can try a new way of doing things as often as I want, and clean up when I've had time to figure out which way works best.

For example, up until recently, the code for a field definition in baseFieldDefinitions() was getting generated in three different ways.

First, the old-fashioned way of doing it line by line, then concatenating the array with a "\n" to make the final code. This is the way most of the old code in DCB was done, but with things that need handling of terminal commas or semicolons, and nesting indents and so on, it was starting to get really clunky.

So then I tried writing something loosely inspired by Drupal's RenderAPI. Because that's a nice big hammer that seems to fit a lot of nails: make a big array of data, chuck your stuff into it, then hand it over to something that makes the output. Except, not so good. Writing the code to make the right sort of array was fiddly. The array of data needed to combine actual data and metadata (such as the class of an annotation), which added levels to the nesting.

Then I hit on an idea: baseFieldDefinitions() fields are a fluent interface, like this:

$fields['changed'] = BaseFieldDefinition::create('changed')
  ->setLabel(t('Changed'))
  ->setDescription(t('The time that the node was last edited.'))
  ->setRevisionable(TRUE)
  ->setTranslatable(TRUE);

What if the code that builds this could be the same, to the point where you could just copy-paste code from, say, the node entity class, and make a few tweaks? Creating the code in DCB would be much simpler, and having the DCB code look like the output code would make debugging easier too.

Using a class with the magic __call() method lets us have just that: a renderer object that treats a method call as some information about code to render. Here's what the builder code for the base field definition code looks like now:

$changed_field_calls = new FluentMethodCall;
$changed_field_calls
  ->setLabel(FluentMethodCall::t('Changed'))
  ->setDescription(FluentMethodCall::t('The time that the entity was last edited.'));
if ($use_revisionable) {
  $changed_field_calls->setRevisionable(TRUE);
}
if ($use_translatable) {
  $changed_field_calls->setTranslatable(TRUE);
}
$method_body = array_merge($method_body, $changed_field_calls->getCodeLines());

It's not yet perfect, as the first line isn't done by this, and the handling of the t() calls could do with some polish; probably by creating a separate class called something like FunctionCall, such that FunctionCall::somefunction() returns the code for a call to somefunction().

But the efficiency and elegance of this approach has led me to devise a new principle for DCB: builder code should look as much as possible like that code that it outputs.

So applying this approach to outputting annotations, the code now looks like this:

$annotation = ClassAnnotation::ContentEntityType([
  'id' => 'cat',
  'label' => ClassAnnotation::Translation("Cat"),
  'label_count' => ClassAnnotation::PluralTranslation([
    'singular' => "@count content item",
    'plural' => "@count content items",
  ]),
]);
$annotation_lines = $annotation->render();

Magic methods used there as well, this time for static calls. The similarity to the output code isn't as good, as annotations aren't PHP code, but it's still close enough that you can copy the code you want to output, make a few simple changes, and you have the builder code.

This work has embodied another principle that I've come to follow: complexity and ugliness should be pushed down, hidden, and encapsulated. Here, the ClassAnnotation and FluentMethodCall have to do fiddly stuff like quoting string values, recurse into nested arrays. They have to handle special cases, like the last line of a fluent call has a semicolon and the last line of an annotation has no comma. All of that is hidden from the code that uses them. That can get on with doing the interesting bits.

There's a great module called the debug module. I'd give you the link… but it doesn't exist. Or rather, it's not a module you download. It's a module you write yourself, and write again, over and over again.

Do you ever want to inspect the result of a method call, or the data you get back from a service, the result of a query, or the result of some other procedure, without having to wade through the steps in the UI, submit forms, and so on?

This is where the debug module comes in. It's just a single page which outputs whatever code you happen to want to poke around with at the time. On Drupal 8, that page is made with:

• an info.yml file
• a routing file
• a file containing the route's callback. You could use a controller class for this, but it's easier to have the callback just be a plain old function in the module file, as there's no need to drill down a folder structure in a text editor to reach it.

(You could quickly whip this up with Module Builder!)

Here's what my router file looks like:

joachim_debug:
  path: '/joachim-debug'
  defaults:
    _controller: 'joachim_debug_page'
  options:
    _admin_route: TRUE
  requirements:
    _access: 'TRUE'

My debug module is called 'joachim_debug'; you might want to call yours something else. Here you can see we're granting access unconditionally, so that whichever user I happen to be logged in as (or none) can see the page. That's of course completely insecure, especially as we're going to output all sorts of internals. But this module is only meant to be run on your local environment and you should on no account commit it to your repository.

I don't want to worry about access, and I want the admin theme so the site theme doesn't get in the way of debug output or affect performance.

If you sometimes want to see themed output, you can add a second route with a different path, which serves up the same content but without the admin theme option:

joachim_debug_theme:
  path: '/joachim-debug-theme'
  defaults:
    _controller: 'joachim_debug_page'
  requirements:
    _access: 'TRUE'

The module file starts off looking like this:

opcache_reset();

function joachim_debug_page() {
  $build = [
    '#markup' => “aaaaarrrgh!!!!”,
  ];

  /*
  // ============================ TEMPLATE


  return $build;
  */

  return $build;
}

The commented-out section is there for me to quickly copy and paste a new section of code anytime I want to do something different. I always leave the old code in below the return, just in case I want to go back to it later on, or copy-paste snippets from it.

Back in the Drupal 6 and 7 days, the return of the callback function was merely a string. On Drupal 8, it has to be a proper render array. The return text used to be 'It's going wrong!' but these days it's the more expressive 'aaaaarrrgh'. Most of the time, the output I want will be the result of dsm() call, so the $build is there just so Drupal's routing system doesn't complain about a route callback not returning anything.

Here are some examples of the sort of code I might have in here.

  // ============================ Route provider
  $route_provider = \Drupal::service('router.route_provider');
  
  $path = 'node/%/edit';
  $rs = $route_provider->getRoutesByPattern($path);
  dsm($rs);

  return $build;

Here I wanted to see the what the route provider service returns. (I have no idea why, this is just something I found in the very long list of old code in my module's menu callback, pushed down by newer stuff.)

  // ============================ order receipt email
  $order = entity_load('commerce_order', 3);
  
  $build = [
    '#theme' => 'commerce_order_receipt',
    '#order_entity' => $order,
    '#totals' => \Drupal::service('commerce_order.order_total_summary')->buildTotals($order),
  ];

  return $build;

I wanted to work with the order receipt emails that Commerce sends. But I don't want to have to make a purchase, complete and order, and then look in the mail logger just to see the email! But this is quicker: all I have to do is load up my debug module's page (mine is at the path 'joachim-debug', which is easy to remember for me; you might want to have yours somewhere else), and vavoom, there's the rendered email. I can tweak the template, change the order, and just reload the page to see the effect.

As you can see, it's quick and simple. There's no safety checks, so if you ever put code here that does something (such as an entity_delete(), it's useful for deleting entities in bulk quickly), be sure to comment out the code once you're done with it, or your next reload might blow up! And of course, it's only ever to be used on your local environment; never on shared development sites, and certainly never on production!

I once read something about how a crucial piece of functionality required for programming, and more specifically, for ease of learning to program with a language or a framework, is being able to see and understand the outcomes of the code you are writing. In Drupal 8 more than ever, being able to understand the systems you're working with is vital. There are tools such as debuggers and the Devel and Devel Contrib modules' information pages, but sometimes quick and dirty does the job too.

Dependency injection is a pattern that adds a lot of boilerplate code, but Drupal Code Builder makes it easy to add injected services to plugins, forms, and service classes.

Now that the Drupal 8 version of Module Builder (the Drupal front-end to the Drupal Code Builder library) uses an autocomplete for service names in the edit form, adding injected services is even easier, and any of the hundreds of services in your site’s codebase (443 on my local sandbox Drupal 8 site!) can be injected.

I often used this when I want to add a service to an existing plugin: re-generate the code, and copy-paste the new code I need.

This is an area in which Module Builder now outshines its Drush counterpart, because unlike the Drush front end for Drupal Code Builder, which generates code with input parameters every time, Module Builder lets you save your settings for the generated module (as a config entity).

So you can return to the plugin you generated to start with, add an extra service to it, and generate the code again. You can copy and paste, or have Module Builder write the file and then use git to revert custom code it’s removed. (The ability to insert generated code into existing files is on my list of desirable features, but is realistically a long way off, as it would be rather complex, a require the addition of a code parsing library.)

But why stop at generating code for your own modules? I recently filed an issue on Search API, suggesting that its plugins could do with tweaking to follow the standard core pattern of a static factory method and constructor, rather than rely on setters for injection. It’s not a complex change, but a lot of code churn. Then it occurred to me: Drupal Code Builder can generate that boilerplate code: simply create a module in Module Builder called ‘search_api’, and then add a plugin with the name of one that is already in Search API, and then set its injected services to the services the real plugin needs.

Drupal Code Builder already knows how to build a Search API plugin: its code analysis detects the right plugin base class and annotation to use, and also any parameters that the constructor method should pass up to the base class.

So it’s pretty quick to copy the plugin name and service names from Search API’s plugin class to the form in Module Builder, and then save and generate the code, and then copy the generated factory methods back to Search API to make a patch.

I’m now rather glad I decided to use config entities for generated entities. Originally, I did that just because it was a quick and convenient way to get storage for serialized data (and since then I discovered in other work that map fields are broken in D8 so I’m very glad I didn’t try to make then content entities!). But the ability to save the generating settings for a module, and then return to it to add to them has proved very useful.

As far as I know, there's nothing (yet) for triggering an arbitrary event. The complication is that every event uses a unique event class, whose constructor requires specific things passing, such as entities pertaining to the event.

Today I wanted to test the emails that Commerce sends when an order completes, and to avoid having to keep buying a product and sending it through checkout, I figured I'd mock the event object with Prophecy, mocking the methods that the OrderReceiptSubscriber calls (this is the class that does the sending of the order emails). Prophecy is a unit testing tool, but its objects can be created outside of PHPUnit quite easily.

Here's my quick code:

  $order = entity_load('commerce_order', ORDER_ID);
 
  $prophet = new \Prophecy\Prophet;
  $event = $prophet->prophesize('Drupal\state_machine\Event\WorkflowTransitionEvent');

  $event->getEntity()->willReturn($order);
 
  $subscriber = \Drupal::service('commerce_order.order_receipt_subscriber');
 
  $subscriber->sendOrderReceipt($event->reveal());

Could some sort of generic tool be created for triggering any event in Drupal? Perhaps. We could use reflection to detect the methods on the event class, but at some point we need some real data for the event listeners to do something with. Here, I needed to load a specific order entity and to know which method on the event class returns it. For another event, I'd need some completely different entities and different methods.

We could maybe detect the type that the event method return (by sniffing in the docblock... once we go full PHP 7, we could use reflection on the return type), and the present an admin UI that shows a form element for each method, allowing you to enter an entity ID or a scalar value.

Still, you'd need to look at the code you want to run, the event listener, to know which of those you'd actually want to fill in.

Would it same more time than cobbling together code like the above? Only if you multiply it by a sufficiently large number of developers, as is the case with many developer tools.

It's the sort of idea I might have tinkered with back in the days when I had time. As it is, I'm just going to throw this idea out in the open.

I started adding unit tests to Drupal Code Builder about 3 years ago, and I’ve been gradually expanding on them ever since. It’s made refactoring the code a pleasant rather than stressful experience.

However, while all generator types are covered, the level of detail the tests go into isn’t that deep. Back when I wrote the tests, they mostly needed to check for hook implementations that were generated, and so quick and dirty regexes on the generated code did the job: match 'mymodule_form_alter' in the generated code, basically. I gradually extended those to cover things like class declarations and methods, but that approach is very much cracking at the seams.

So it’s time to switch to something more powerful, and more suited to the task.

I’ve already removed my frankly hideous attempt at verifying that generated code is correctly-formed PHP, replacing it with a call to PHP’s own code linter. My own code was running the generated PHP code files through eval() (yes, I know!) to check nothing crashed, which was quick and worked but only up to a point: tests couldn’t create the same function twice, as eval()ing code that contains a function declaration brings it into the global namespace, and it didn’t work at all for classes where while tests were being run, as the parent classes in Drupal core or contrib aren't present.

It's already proved worthwhile, as once I'd converted the tests, I found an error in the generated code: a stray quote mark in base field definitions for a content entity, which my approach wasn't picking up, and never would have.

The second phase is going to be to use PHPCS and Drupal Coder to check that generated code follows Drupal Coding Standards. I'm currently getting that to work in my testing base class, although it might be a while before I push it, as I suspect it's going to complain about quite a few nipicks in the generated code that I'll then have to spend some time fixing.

The third phase (this is a 3-phrase programme, all the best ones are) is going to be to look into using PHP-Parser to check for the presence of functions and methods in the code, rather than my regex-based approach. This is going to allow for much more thorough checking of the generated code, with things such as method order in the code, service injection, and more.

After that, it'll be back to some more refactoring and code clean-up, and then some more code generators! I have a few ideas of what else Drupal Code Builder could generate, but more ideas are welcome in the issue queue on github.