PHP on PHP Boy Scout

Free Open Source Website for Scouts

Mon, 12 Jan 2015 00:00:00 +0000

I’ve been a Scout Leader for a few years now and the District I work within have very little by way of internet presence. As a bit of a pet project I started building a simple Scout based website for them to use.

Its nothing too fancy, I created a simple module and theme for the Silverstripe CMS and have now put it into a GitHub Repository to share with the wider scouting community.

I chose Silverstripe because of the speed with which I could develop something usable as well as providing a super simple management interface that can be handled by users of all skill levels.

The module itself extends some very common extensions available for the CMS and makes them scout focused. Features include.

Customisable theme
Multi tiered Event Calendars
Customisable Group/Section Pages
Dynamic Forms
A reliable News/Blog system

These are just a few of the most obvious features and hopefully I will continue to add more.

I’m also offering to help any Scout Groups/Districts/Counties if they are wanting to use these modules and get their sites built and up and running for them free of charge.

If you want to take a look at the code and have a play yourselves you can find it online at

If you want to get in touch or would like more information about having a website built for you please fill in the form below.

[contact-form to='matt@phpboyscout.uk’ subject=‘Request for Scouts Website’][contact-field label=‘Name’ type=‘name’ required=‘1’/][contact-field label=‘Email’ type=‘email’ required=‘1’/][contact-field label=‘Scout Group/District/County’ type=‘url’ required=‘1’/][/contact-form]

Introducing ZuQ - A Simple ZeroMQ Queuing Daemon

Tue, 19 Mar 2013 00:00:00 +0000

We recently had the need to create a queuing system to replace an implementation of RabbitMQ that was being used on a previous project. The reasoning behind this is that the requirements of the project required a very custom implementation of a queuing system that would drastically alter in architecture as the project grew and RabbitMQ just wasn’t going to fit the bill. However to start with we required something super simple and efficient that could be expanded and developed as required. After a little investigation and a lot of recommendation from others we decided to use ZeroMQ as our transport layer for that very reason, as we could build something which could span across multiple servers and was fast.

The diagram above helps describe our basic queuing system. We have a queue daemon that is continuously listening for connections and two clients, one that populates the queue and the other that retrieves from the queue.

The Clients

Each client is written in PHP and uses a 0mq socket to communicate with a service, in this case our queue service. We used a SOCKET_REQ type of socket in order to have a request/response communication with our queue service.

function client_socket(\ZMQContext $context)
{
 // SOCKET_REQ used to create a client that sends requests to and receive from a service
 $client = new \ZMQSocket($context,\ZMQ::SOCKET_REQ);
 $client->connect("tcp://localhost:5555");
 // SOCKOPT_LINGER = 0 Configure socket to not wait at close time
 $client->setSockOpt(\ZMQ::SOCKOPT_LINGER, 0);
 return $client;
}

public function injectIntoQueue()
{
 $context = new \ZMQContext();
  $client = $this->client_socket($context);
  $msg = "This is a message";
  $retries_left = 3;
  $read = $write = array();
  while ($retries_left) {
    // We send a request, then we wait to get a reply
        $client->send($msg);
        $expect_reply = true;
        while ($expect_reply) {
        // Poll socket for a reply, with timeout
            $poll = new \ZMQPoll();
            $poll->add($client, \ZMQ::POLL_IN);
            $events = $poll->poll($read, $write, 2500);
            // If we got a reply, process it
            if ($events > 0) {
            // We got a reply from the server, must match sequence
                $reply = $client->recv();
                if (intval($reply) == $msg) {
                $retries_left = 0;
                    $expect_reply = false;
                }
             } elseif (--$retries_left == 0) {
            break;
             } else {
               // Old socket will be confused; close it and open a new one
                $client = $this->client_socket($context);
                // Send request again, on new socket
                $client->send($msg);
 }
 }
 }
}

You can see from the code above, have a 3 strike rule. The reasoning behind this is that if the client fails to connect to the queue service more than 3 times, we can stop trying to inject into the queue and move on to the next item. As we ultimately intend to adapt the lazy pirate pattern we have made it so that if the socket times out, we can then create a new socket and retry. Without this, as the architecture becomes more complicated we may then end up in a situation where we might have errors, thus the recommend solution is to create a new socket. Once the client has sent its message to the queue, we poll for a response (i.e. which is the message we sent returned back). Once we have a response that is valid, meaning that the queue has been populated, we can stop polling until the next message.

The Frontend Client

protected function getFromQueue()
{
 $context = new \ZMQContext();
 $worker = new \ZMQSocket($context, \ZMQ::SOCKET_REQ);
 $read = $write = array();
 // Set random identity to make tracing easier
 $worker->connect("tcp://localhost:5556");
 // Tell queue we're ready for work
 $worker->send("ready");
 $reply = $worker->recv();
 return $reply;
}

Our Frontend Client is much simpler as it is part of a process that is being continually updated, therefore it doesn’t need the same connection retries are the Backend Client. We simply send “ready” to the queue system and if the queue is populated it will return us the first item.

The Queuing Service

This is a continuously running executable created using c++.

zmq::context_t context(1);
zmq::socket_t frontend (context, ZMQ_ROUTER);
zmq::socket_t backend (context, ZMQ_ROUTER);
backend.bind("tcp://*:5555");
frontend.bind("tcp://*:5556");

We create two sockets of type ZMQ_ROUTER which is an advanced pattern used for extending request/reply sockets. This means when we improve our queuing system we will be able to route packets to specific recipients using an address in the message.

After creating our sockets, we initialise them

// Initialize poll set
zmq::pollitem_t items [] = {
 { frontend, 0, ZMQ_POLLIN, 0 },
 { backend, 0, ZMQ_POLLIN, 0 }
};

//poll the sockets - this seems to poll both sockets at the same time
zmq::poll (items, 2, -1);

Backend Handler

If we get a message from the backend, we check the contents to see if it contains purge at which point we empty the queue, otherwise we push the msg contents onto the queue. Finally we send the message back to the backend to show that the message has been received.

//receive msg from client
if (items [1].revents & ZMQ_POLLIN) {
 //get message from client
 zmq::message_t message(0);
 string client_addr = s_recv (backend);
 string empty = s_recv (backend);
 assert (empty.size() == 0);
 string msg = s_recv (backend);

 //allow the backend to purge the queue
 if(msg == "purge") {
 while (!queue.empty()) queue.pop();
 } else {
 queue.push(msg);
 }

 //send response back to the backend
 s_sendmore (backend, client_addr);
 s_sendmore (backend, "");
 s_send (backend, msg);
}

Frontend Handler

If we get the “ready” message from the frontend client, we pop a message off the queue and return it to the frontend client. If the queue is empty we send an “empty” message back instead.

// Handle activity on frontend
if (items [0].revents & ZMQ_POLLIN) {
 //get message from worker
 zmq::message_t message(0);
 string worker_addr = s_recv (frontend);
 string empty = s_recv (frontend);
 assert (empty.size() == 0);}
 string msg = s_recv (frontend); string queueMsg;
 if(msg == "ready") {
 if(queue.size() > 0) {
 queueMsg = queue.front();
 queue.pop();
 } else {
 queueMsg = "empty";
 }
 }
 //send reply to worker with contents of queue
 s_sendmore (frontend, worker_addr);
 s_sendmore (frontend, "");
 s_send (frontend, queueMsg);
}

That is our complete queuing system using ZeroMQ with PHP and C++.

Summary

Using the above has allowed us to create a very simple in memory queue daemon that we can use to quickly pass data from one system to a another. On the whole it works well and we are looking to expand on it in the near future to increase both its functionality and scalability.

You can find the queueing daemon (christened as “ZuQ”) on github @ https://github.com/zucchi/ZuQ

Docblock, Oh Docblock, wherefore art thou Docblock (hint: Zend Optimizer Plus lost them)

Fri, 01 Mar 2013 00:00:00 +0000

tl;dr> I make a terrible assumption about Zend Optimizer+ and am corrected by Dominic in the comments;

Terrible post title I know but its the best I could come up with.

I’ve just come up for air after spending the majority of the day debugging some issues on our current development sandbox.

Now our sandbox tends to be quite bleeding edge in some circumstances and as such we run a fair few bits of unstable code. On the sandbox in question we have been running PHP 5.4.11 and unfortunately we have struggled to get APC working with it just the way we need it to. The lack of APC tends to make this sandbox quite slow.

We recently saw that Zend have open-sourced their OptimizerPlus extension (https://github.com/zend-dev/ZendOptimizerPlus) and that it was compatible with 5.4…. Fantastic, or so we thought.

So I added the new OptimiserPlus to the sandbox and everything was going swimmingly. That was until we had to run one of the utility scripts that we use to rebuild some of our data structures. These scripts make use of different parts of both Zend Framework and Doctrine which tend to rely on some heavy DocBlock annotations.

Now having used both APC and Zend Server knowing that they done affect this kind of functionality I had expected that OptimizerPlus would be fine…. Wrongo. It took me a good few hours of head scratching trying to figure out what had happened.

It turns out that OptimizerPlus suffers from the same flaws that eAccellerator does and strips Docblocks when caching the bytecode. This results in Reflection returning false when you call methods such as `getDocComment()`.

All in all its not the end of the world I just disable OptimizerPlus and have to wait till I can get APC working. Not my ideal scenario but I can live with it.

Something that does concern me is that there is currently an RFC that has gone to vote (https://wiki.php.net/rfc/optimizerplus) about integrating OptimizerPlus into the PHP 5.5 distribution. While this is great I do worry how many other things may break and will they be picked up and fixed for the 5.5 release.

Update: Since writing this post the RFC has finished being voted upon and has been approved. You can expect to see Optimizer Plus appearing bundled with PHP soon.

Update (15th Mar 13): Thanks to Dominics’ comment I now know that you can tell Optimizer+ to retain your Docblocks by setting your config using

zend_optimizerplus.save_comments (default "1")
	If disabled, all PHPDoc comments are dropped from the code to reduce the
	size of the optimized code. Disabling "Doc Comments" may break some
	existing applications and frameworks (e.g. Doctrine, ZF2, PHPUnit)

zend_optimizerplus.load_comments (default "1")
	If disabled, PHPDoc comments are not loaded from SHM, so "Doc Comments"
	may be always stored (save_comments=1), but not loaded by applications
	that don't need them anyway.

That’ll teach me to write a blog post without investigating more first.

Debug PHP CLI on Remote Server with Xdebug and PHPStorm

Wed, 06 Feb 2013 00:00:00 +0000

This was a head scratcher when I ran into this yesterday and I thought I would share my solution to the following scenario:

I need to debug PHP Command Line script, located on Remote LAMP Virtual WebServer running in Virtual Box with a Shared Folder, using local PHPStorm 5.0.

The solution:

You first must set PHPStorm to use remote file paths. To set these go to the following:

PHPStorm -> Peferences -> PHP -> Servers

This gives the following display:

Replace the Name, Host and Absolute path on the server, to match your own settings. Note keep the Name and Host the same for ease.

Next add some breakpoints in PHPStorm and set it to listen for any debug connections using the listener icon:

Now login to your Remote Server via SSH etc.

You now need to change settings for Xdebug in either xdebug.ini or php.ini depending on how you installed it. You also need to know the IP of the local machine. This can permanently set in the Network Setting of your VM in Virtual Box, so you will never have to change it. In my example the local machine running PHPStorm is:

192.168.56.1

Now edit the ini file that contains your Xdebug settings and set the following:

xdebug.remote_host = 192.168.56.1
xdebug.remote_connect_back = 0
xdebug.remote_port = 9000
xdebug.remote_handler = dbgp
xdebug.remote_mode = req
xdebug.remote_enable = 1
xdebug.idekey = phpstorm1

Be aware you might have to change the remote_host and the idekey based on your own environment. To better understand what each option does, see Xdebug Settings

Finally, when running the script you must set the following variables:

PHP_IDE_CONFIG="serverName=dev.example.com"

PHP_IDE_CONFIG will tell PHPStorm how to map the Remote File Paths to what it sees Locally. Again replace the URL with the Name/Host you set in PHPStorm. Note: You can export this, if your system is only running one site; mine is not.

You can run this inline with your script:

PHP_IDE_CONFIG="serverName=dev.example.com" ./testscript.sh

This should send you to PHPStorm where you earlier placed breakpoints.

Happy Debugging!

Compiling PHP 5.4 on Ubuntu 12.04

Tue, 06 Nov 2012 00:00:00 +0000

So recently I’ve been working with PHP 5.4 a LOT. Unfortunately Ubuntu (my main dev environment) is behind the times. So I’m resorting to compiling PHP manually.

Not a daunting as it may first appear. The really tricky part is working out your dependencies and `configure` script.

Hence the reason for this post as a reminder for myself and others that may want to do a quick compile. (I would recommend that if your compiling for a production/live environment that you make sure you understand what it is your compiling though before just using what’s here)

So where to start. Dependencies first I think

Ubuntu allows you to install dependencies for building source `apt-get build-deps`. We will use this and install any extras we may need.

apt-get install \
libxml2 \
libxml2-dev \
libssl-dev \
pkg-config \
curl \
libcurl4-nss-dev \
enchant \
libenchant-dev \
libjpeg8 \
libjpeg8-dev \
libpng12-0 \
libpng12-dev \
libvpx1 \
libvpx-dev \
libfreetype6 \
libfreetype6-dev \
libt1-5 \
libt1-dev \
libgmp10 \
libgmp-dev \
libicu48 \
libicu-dev \
mcrypt \
libmcrypt4 \
libmcrypt-dev \
libpspell-dev \
libedit2 \
libedit-dev \
libsnmp15 \
libsnmp-dev \
libxslt1.1 \
libxslt1-dev

And now the configure

./configure \
--prefix=/usr/local/php \
--with-apxs2=/usr/local/apache2/bin/apxs \
--enable-fpm \
--with-fpm-user=www-data \
--with-fpm-group=www-data \
--with-config-file-path=/usr/local/php/conf \
--with-config-file-scan-dir=/usr/local/php/conf.d \
--enable-debug \
--with-openssl \
--with-kerberos \
--with-zlib \
--enable-calendar \
--with-curl \
--with-curlwrappers \
--with-enchant \
--enable-exif \
--enable-ftp \
--with-gd \
--with-jpeg-dir=/usr \
--with-png-dir=/usr \
--with-vpx-dir=/usr \
--with-freetype-dir=/usr \
--with-t1lib \
--enable-exif \
--enable-gd-native-ttf \
--enable-gd-jis-conv \
--with-gettext \
--with-gmp \
--with-mhash \
--enable-intl \
--enable-mbstring \
--with-mcrypt \
--with-mysql \
--with-mysqli \
--enable-pcntl \
--with-pdo-mysql \
--with-pdo-pgsql \
--with-pgsql \
--with-pspell \
--with-libedit \
--with-readline \
--enable-shmop \
--with-snmp \
--enable-soap \
--enable-sockets \
--enable-sysvmsg \
--enable-sysvshm \
--with-xsl \
--enable-zip \
--with-pear \
--enable-zend-signals \
--enable-maintainer-zts

Once these are done then we follow the standard make process. Notice we are also running make test… very important as it givges more data for the developers to work with.

make && make test && make install

The next thing is configuring your php.ini file as the install doesn’t have one yet so we copy either the production or development default from the source code to the new conf dir and edit to suit your needs.

cp {php-source-dir}/php.ini-(development|production) /usr/local/php/conf

Thats it. All ready to roll… almost, this installation is the one I use for use with a webserver so you will want to add the appropriate directives to apache.

LoadModule php5_module modules/libphp5.so

AddHandler php5-script .php
AddType text/html .php