Pimple Free

I recently started a new web app project at work. Silex was chosen as the PHP framework we'd be using. Silex makes heavy use of Pimple, a very small dependency injection container that allows you to map factories or values to string keys on an object that implements PHP s ArrayAccess interface. Here's a quick example of how one might use Pimple:

$pimple = new \Pimple\Container();

$pimple['db.client'] = function($pimple) {
    return new MysqlClient();
};

// elsewhere in project...
$usersDao = new UserDao($pimple['db.client']);

Seems innocent enough, but theres a few things wrong.

Does db.client exist? I guess we'll find out at runtime! If it does exist, where is it defined? Silex and Pimple promote a Service Provider pattern, so the code above would actually look a bit like this:

class DbServiceProvider implements ServiceProviderInterface
{
    public function register(Container $pimple)
    {
        $pimple['db.client'] = function($pimple) {
            return new MysqlClient();
        };
    }
}

class UserServiceProvider implements ServiceProviderInterface
{
    public function register(Container $pimple)
    {
        $pimple['data.user'] = function($pimple) {
            return new UserDao($pimple['db.client']);
        };
    }
}

$pimple->register(new DbServiceProvider());
$pimple->register(new UserServiceProvider());

This is easy to follow and all the Service Providers in a Silex app live in a single directory so you know where to look. That said, Silex defines quite a few factories for Pimple. I came up with a regular expression search to find out roughly how many. Silex always refers to Pimple as $app and always uses array access, so the statement below looks to that with the assignment operator ("="). The second command is similar except that it looks for keys that have variables in them. These ones are usually in loops, so the assignments are multiplied by the number of iterations. It turns out we're looking at a baseline of around 150-200 assignments. That's a bit of a load on the mind and we haven't even gotten into how the app actually works.

ack '\$(this->)?app\[.*?]\s*=' vendor/silex/silex/src | wc -l    
154
ack '\$(this->)?app\[.*?\$.*?]\s*=' vendor/silex/silex/src | wc -l
15

Undoubtedly we have lots of Pimple keys and factories to keep in our head. To further complicate matters the return types from these factories is never explicitly declared. In my example, what type should we expect for db.client? If you look at the factory that creates it you might think its a MysqlClient, but suppose MysqlClient implements DbClient. So what is the guaranteed interface of db.client - a DbClient or MysqlClient? Unfortunately, nothing is guaranteed. Your best bet is searching for where the key is used and what type is expected there.

An IDE that could interpret this code could offer up tooltips, autocomplete, let you jump to declarations, and much more. Surely, this would accelerate our understanding of the system. Sadly, this doesn't seem possible using Pimple. There could be more than one instance of Pimple, so your IDE would need to know which one you were inspecting and it can't do that without running your code. Additionally, Pimple's use of ArrayAccess, which results in a single method returning mixed values based on a method parameter, cannot be property documented using IDE-supported PHPDoc tags.

Given my complaints about Pimple I wondered what an alternative implementation might be. There are other PHP Dependency Injection Containers but most are not nearly as simple as Pimple (one file, couple hundred lines). Here's what I've come up with. The Container class is really the only library code.

class Container
{
    /** @var null|Container */
    protected $rootContainer = null;

    public function __get($name)
    {
        return $this->{'_' . $name}();
    }

    protected function child(Container $container)
    {
        $container->rootContainer = $this->rootContainer ?: $this;
        return $container;
    }
}

/**
 * @property-read ApiContainer $api
 */
class AppContainer extends Container
{
    protected function _api()
    {
        return $this->child(new ApiContainer());
    }
}

In this example, AppContainer is the base container for your application. It has one child container, ApiContainer. A user would get an ApiContainer instance by calling AppContainer::api. In this simple example there isn't much benefit in using __get versus just calling a public api() method. In a full implementation you'd likely have code in __get to do thing like provide singleton functionality or maybe other features you see in Pimple's offsetGet (we'll see some of this later).

Pimple gets its configuration at runtime. Until then, our IDEs have no idea what instance of Pimple is being operated on and no idea what configuration it might hold. This isn't beneficial at all. Obviously, not all instances of Pimple are the same. Our application can't just use any instance interchangeably. Our application needs certain keys to exist and for those keys to return values of certain types. Our classes have certain type needs and we know what they are before runtime.

My example takes advantage of that fact. AppContainer is a distinct class with a distinct interface. It always has an api property of type AppContainer. I tell my IDE this using the property-read PHPDoc tag.

Here's an expanded usage example:

/**
 * @property-read ApiContainer $api
 * @property-read HttpDriver $httpDriver
 */
class AppContainer extends Container
{
    protected function _api()
    {
        return $this->child(new ApiContainer());
    }

    protected function _httpDriver()
    {
        return new CurlDriver();
    }
}

/**
 * @property AppContainer $rootContainer
 * @property-read ApiClient $client
 */
class ApiContainer extends Container
{
    protected function _client() {
        return new ApiClient($this->rootContainer->httpDriver);
    }
}

interface HttpDriver {}

class CurlDriver implements HttpDriver {}

class ApiClient
{
    public function __construct(HttpDriver $httpDriver)
    {
        $this->httpDriver = $httpDriver;
    }

    public function get($url) {
        // ...
    }
}

$app = new AppContainer();
$app->api->client->get('http://www.google.com');

This works. IDE's like IntelliJ/PHPStorm can use the PHPDoc tags to determine what type of objects you're working with the whole way through. You get autocomplete. You can jump to declarations of relevant types. MISSION ACCOMPLISHED! Uhh, theres just one problem (that I can think of). Pimple will allow you to easily change out factory methods for use in testing, but my example code will not. My Container class would not let you do this:

$app->api->client = MockApiClient();
var_dump($app->api->client);

It won't error out, but $app->api->client will still be a ApiClient in that var_dump call instead of being a MockApiClient. The reason is that after setting the value to MockApiClient our instance of ApiContainer is no longer referenced. The next attempt to access $app->api->client will generate a new ApiContainer with a new ApiClient as both are transient objects.

Without too much work we can add support for this. Below, I've added the concept of values and factories to Container. I've updated AppContainer to register ApiContainer as child which we now always store as a Singleton Factory. This means that once $app->api is accessed we hold the instance of ApiContainer for future accesses, thereby solving the above-mentioned transient object problem. Finally, I've used the value feature to hold and return a reference to the MockApiClient object.

class Container
{
    /** @var null|Container */
    protected $rootContainer = null;

    /** @var callable[] */
    protected $factories = [];

    /** @var mixed[] */
    protected $values = [];

    public function __get($name)
    {
        if (isset($this->values[$name])) {
            return $this->values[$name];
        } elseif (isset($this->factories[$name])) {
            return $this->factories[$name]();
        }

        $result = $this->{'_' . $name}();

        if ($result instanceof Singleton) {
            $this->factories[$name] = $result;
            $result = $result();
        }

        return $result;
    }

    public function __set($name, $value)
    {
        if (is_callable($value)) {
            $this->factories[$name] = $value;
        } else {
            $this->values[$name] = $value;
        }
    }

    protected function child(Container $container)
    {
        $container->rootContainer = $this->rootContainer ?: $this;
        return new Singleton($container);
    }
}

class Singleton
{
    public function __construct($instance)
    {
        $this->instance = $instance;
    }

    public function __invoke()
    {
        return $this->instance;
    }
}

$app->api->client = MockApiClient();

This simple system should enable the user to take advantage of a powerful IDE and help them understand a system that uses a Dependency Injection Container more easily than a Pimple-based system. I'll update this post shortly once I've created a Composer package for the code in this article.

UPDATE: Check Pedic out on Github!

Remote PHPUnit execution using Intellij or PHPStorm

Trying to get Intellij or PHPStorm setup to remotely run PHPUnit with Xdebug can be a bit of a beast. The tools used to run PHPUnit remotely seem to make the assumption that your web accessible directory is also where your tests and the rest of your code lives. This assumption is often false and leads to problems when mapping local paths to remote paths. Additionally, Intellij's configuration can be a bit daunting. All of my notes below came after much trial and error.

Prerequisites

  • Installed Intellij or PHPStorm
  • Installed PHP Remote Interpreter Intellij plugin (may not be required with PHPStorm?)
  • Remote host with web server with PHP and Xdebug support. Nginx with PHP-FPM are used in these examples, though the strategy should easily port to Apache with mod_php.
  • PHPUnit installed on the same remote host. PHPUnit can be installed globally or per-project using Composer. My examples take the later approach.
  • PHPUnit tests to run in your project

The following steps worked for me using Intellij IDEA 14.0.2. My project files are in a directory mounted by the remote host. Other means of deploying the files may be used but that may require you to vary from this setup. Though personally I am just using a manually setup virtual machine running in Virtualbox, this is still applicable if you manage you virtual machine using Vagrant other development environment manager.

Overview

Here is the basic flow of the remote execution of PHPUnit tests triggered by Intellij and subsequent debugging of that request:

  • User tells Intellij to listen for Xdebug connections
  • User triggers a unit test run from Intellij
  • Intellij deploys a file named _intellij_phpunit_launcher.php to the remote host at a path that is accessible via HTTP
  • Intellij requests _intellij_phpunit_launcher.php over HTTP, passing along configuration in the query string
  • _intellij_phpunit_launcher.php triggers your test code until a debugger breakpoint is hit (the first executed line of PHP may be the debugger breakpoint)
  • XDebug determines where the debugger client is using the $_SERVER['REMOTE_ADDR'] value from the web request and the configured xdebug.remote_port value (this is assuming you're using xdebug.remote_connect_back)
  • Intellij, which is still listening for connection, recieves a request from Xdebug to initiate a debugging session
  • As the user walks through code using the Intellij debugger client, Intellij sends commands to Xdebug using the DBGp protocol (details on the communication and the roles of Intellij and Xdebug can be seen here - Xdebug 2 Remote Debugging Communication Set-up)
  • The user completes their debugging session
  • Unit test results are returned as the body of the original HTTP request

Xdebug

First, the easiest part - configuring Xdebug. Remember, this is set on your remote host, not locally. Setting up PHPUnit with Xdebug locally is possible, but done differently, and I won't be covering it here.

zend_extension=xdebug.so
xdebug.remote_enable=1
xdebug.remote_connect_back=1
xdebug.remote_port=9000
xdebug.remote_handler="dbgp"
xdebug.remote_autostart=1
xdebug.idekey="PHPSTORM"

The interesting options here are...

  • xdebug.remote_connect_back - This tells Xdebug to connect to the host that initiated the HTTP request which triggered Xdebug
  • xdebug.remote_port - This is the port at which your IDE will listen for Xdebug to initiate a session
  • xdebug.idekey - Needs to match the key set in your IDE

More on Xdebug settings can be found here.

Nginx

The Nginx configuration isn't bad either, but requires some explanation. Add this code as an Nginx site (Usually in the sites-enabled directory that the main nginx.conf file includes). Remember this configuration is for access to your project for testing only. You likely only want to use this configuration in your development environment and not in your production environment.

server {
    listen 81;
    server_name foo;
    root /foo;        

    location = /_intellij_phpdebug_validator.php {
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

    location = /_intellij_phpunit_launcher.php {
        set $args $args&load_mode=l&load_path=$document_root/vendor/autoload.php;
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PHP_VALUE "display_errors=1\nhtml_errors=0";
        fastcgi_read_timeout 3600;
        include fastcgi_params;
    }
}
  • Lets say your web accessible directory of you project is /foo/web, served at port 80, so in this example your project root is just /foo, served at port 81.
  • _intellij_phpdebug_validator.php works similarly to how _intellij_phpunit_launcher.php but instead of returning unit test results its for supporting Intellij's "Validate remote evironment" feature, which checks for the existance of a php.ini file with Xdebug configured.
  • For the _intellij_phpunit_launcher.php location block first we're adding some extra query parameters. Setting load_mode to l and load_path to vendor/autoload.php causes PHPUnit to be loaded based on your composer autoloader configuration rather than just using a globally installed version. There might be another way to do this, but I couldn't find it.
  • Enabling display_errors will cause any error messages to show up in the Run output panel rather than failing silently.
  • You may have configred Xdebug to override normal error output with HTML formatted out which in this context makes it completely unreadable, so disable it.
  • Unit test execution will timeout after 30 seconds by default. You'll easily exceed 30 seconds in most debugging sessions, so we set the timeout to an hour instead.

If you want to know exactly how Intellij is interacting with your web server check your Nginx access and error logs. If you want to know what code its executing just comment out the _intellij_phpunit_launcher.php location block, restart nginx, and run your tests from Intellij (using the configuration I'll get to later). The file will be served as a plain text file and show in your Run panel output.

Intellij

Interpreter configuration

  • In your Intellij preferences go to Language & Frameworks > PHP > PHPUnit > Interpreter > "..."
  • Enter in server settings and use the "Test connection..." button to verify your settings are correct

Xdebug client configuration

  • In your Intellij preferences go to Language & Frameworks > PHP > Debug
  • In the Xdebug section enter port 9000 and check "Can accept external connections"

Run configuration, deployment server, debug server

  • In Intellij go to Run > Edit configurations... > "+" > PHPUnit on Server
  • Name your configuration - I call mine "All tests"
  • Set Test to "XML file"
  • Check "User alternate configuration file" and select your phpunit.xml or phpunit.xml.dist file
    • Click Server > "..." > "+"
  • Name your deployment server - Following my examples here I'd call mine "foo:81"
  • Set Type to "Local or mounted folder" and set the Web server root URL to "http://foo:81"
  • Click the Mappings tab
  • Select the local project root path that is mounted to /foo on the remote host
  • Set Deployment path on server to "." and Web path on server to "/"
  • Click OK
  • Make sure your new server is selected on the Servers menu back on the Run/Debug configurations window
  • Click the Remote tab > Debug server > "..." > "+"
  • Set name to something like "foo:81", Host to "foo", Port to "81"
  • Add a path mapping from your local project root to your remote project root
  • Click Validate remote environment > Validate, validation should succeed and list some information about the configuration of the remote host. If it fails the server configuration for _intellij_phpdebug_validator.php did not work, so check your web server logs to find out what went wrong.
  • Close the Validate Remote Environment window
  • Click OK

Run it!

  • In Intellij go to Run > Start Listening for PHP Debug Connections
  • Run > Run... > "All tests"
  • Accept the incoming connection
  • Hit the play button on the Debug panel, as it likely is on the first line of the _intellij_phpunit_launcher.php file. You might be able to add a mapping for the file, but it doesn't always work for me and I likely don't care about debugging from their anyways.

Other resources

When in Rome

There is something about programming conventions that gets everyone riled up. Everyone has an opinion and no one agrees. Today, I got a little riled up. I'd like to lay on your couch and tell you about it while you sit attentively taking notes with a quizzical expression on your face. You'll ask me about my recurring dreams and my relationship with my father.

Um, right. My employer's product has a Python component that I've been working on more and more recently. In it, I wrote some code like this:

'I like to eat %s' % food

I was told that this is the wrong way and the better way is...

'I like to eat {}'.format(food)

Why is the second way better than the first? I was told that the reason is readability and also that it was part of the company's Python coding standard. Hmmm. I care a lot about readability, but... really? If I had multiple replacement arguments I'd switch to format() and provide some nicely named tokens, though you could still do it using the % operator.

'I like to eat {food} for {meal}'.format(food='bacon', meal='all meals')
'I like to eat %(food)s for %(meal)s'. % {'food': 'bacon', 'meal': 'all meals'}

It turns out that format() is Python's recommendation (str.format... is the new standard in Python 3, and should be preferred to the % formatting) and thats good enough for me. My coworker was right, but for the wrong reason. Readability isn't a good reason here and neither is the company standard.

The value of decisions

In an earlier article I wrote that programming isn't an art, and the value of our decisions is not subjective. So, what is the value of this decision? Keep in mind this is proprietary software, its forever-bound to Python 2 unless some epic refactoring is performed, and, lets be honest, any python developer could read either version very quickly.

There is another important distinction: This is implementation not interface, meaning it will never be reused (unlike an interface), so even if it had a defect its effect would not be multiplied.

Hopefully, I've convinced you that in this context the value of this decision is about $0. It doesn't add technical debt and it also doesn't save us money later. Its a wash.

Another coworker, having witnessed my little scuff, kindly commiserated with me. He had gotten a slap on the wrists for not following the company's PHP class naming standard. We use camelcase and when the class name contains acronyms we only capitalize their first letter (ex. Http, not HTTP).

It was a Friday around 4pm and our project was ready on schedule (high five!), so I had time and maybe I was in a disagreeable mood, but I had to argue again. His decision could be a cause for error for many future developers.

In PHP, how many times have you forgotten which order $needle and $haystack are in in the array functions (its inconsistent)? In Python, do you find that after writing some tests are you creating method names like getById instead of get_by_id (because unittest.TestCase methods use camelcase, unlike most other python functions)?

Conventions allow us to create mental shortcuts. One can assume things about a well designed piece of code without even looking at it. When you correctly assume things you save time, and saving time is saving money.

The language of design patterns has a similar time-saving effect. If I see window.decorator.Scrollbar or storage.adapter.Redis being used, I already know something about the class, maybe even a lot about it if I've used other window.decorator's or storage.adapter's. I know the problem the code needed to solve and how it was solved. I can make a lot of assumptions and save a lot of time.

Using HTTP versus Http in a class name in this context isn't an expensive defect. Its cost is not $0, but I'd say it's pretty low. It is multiplied though. This class name will be referenced in multiple places by multiple developers and each will have a chance to fall victim to this inconsistency. There is an advantage to following the established convention and no disadvantage, so... when in Rome...

Fiddle while Rome burns

There's one more aspect of this experience that burns a little more than the others. Coding standards can provide some value, but they don't imply code quality one bit. Spend a few minutes on github and I'm sure you can find a plethora of PEP8-compliant garbage.

Its easy for these low-value matters of style to dominate code reviews, which really is a shame. What about the high-value decisions? If all code is bad, how could this possibly be part of the solution? If you care about a standard like PEP8, set up a PEP8 validator in your build process and be done with it. Knowing the ins and outs of these standard shouldn't make anyone feel useful. Knowing how to take a software design and evaluate it based on the business needs you've identified... thats value.

Page 1 / 9 »