We build innovative web based applications

This article assumes you have already experimented with using the vim debugger as described in this fine article. Big respect to Sueng Woo Shin for writing the debugger script itself. If you've already experimented with it and think you want to use it more then read on...

Using a debugger wins hands down over the good old echo/die approach but if you want to use the aforementioned debugger (which you do because you want to use vim right? right!) then you may need to get used to a few little quirks and maybe even tweak it a little bit so feels a bit....righter.

So with that in mind here's a few little things that I've done and realised that have made it start to feel that little bit righter to me ...

How to debug a script which is invoked via the PHP CLI

To debug a page being served up via apache you point your browser to:

http://example.com/index.php?XDEBUG_SESSION_START=1

in order to set a cookie your browser and then the presence of this cookie indicates that you're keen to debug some stuff.

If the code you want to debug is being run via the PHP CLI then the above won't work so instead you need to set an environment variable before you run invoke the PHP CLI to run your script. This works for tcsh:

setenv "XDEBUG_CONFIG=vim=1"

Then run your script via the CLI (eg, php -f my_script.php) and as long as you have Vim open and have pressed F5 (like you normally do when you want to debug in vim) then you should be able to debug your script.

What's going on with regards to the files that get opened when a debugging session starts up?

Well regardless of if you like it or not the debugger always opens the "first" file in the call stack, if you like. So if you are debugging a handler that gets invoked from say http://workingsoftware.com.au/myapp/index.php?h=FindProduct then it always opens index.php. This is quite annoying cos most of the time you don't really want to debug index.php...in the above example you probably want to debug find_product.class.php, which is the file where the application code actually resides (whereas index.php is part of a framework and you're usually interested in debugging the application, not the framework). (ed: this is referring to usage of the RocketSled PHP framework but will be similar for any framework where you have one script which dispatches control to a Controller or Handler class or script)

There's no real way around this that I can see. So this is what I do if say I want to debug a file called find_product.class.php which is a Handler class (or Controller in MVC terminology) invoked by hitting the URL http://workingsoftware.com.au/myapp/index.php?h=FindProduct:

1. cd myapp/packages/myapp/handlers/
2. open find_product.class.php at the command line by doing: vim find_product.class.php
3. hit F5
4. quickly (within the 5 second limit....see below - I have made this limit longer) go to the URL in your browser (ie, http://workingsoftware.com.au/myapp/index.php?h=FindProduct)
5. the debugger will open up index.php and in the process close your find_product.class.php leaving you with index.php and the other debugger specific windows. Bit annoying. Since you dont want to debug index.php close this file by doing :q
6. now you're left with just the debugger specific windows (actually viewports) in vim. Now open up the file you want to actually debug, find_product.class.php, by doing :sp find_product.class.php
7. Now go ahead and do your debugging. Once the debugger session finishes you'll be left with the find_product.class.php file open in vim.

The good news is that if you set any breakpoints in your find_product.class.php file while you were debugging, as long as you don't close vim itself (doesn't matter if you close the find_product.class.php file and reopen it, as long as vim stays running) then you can hit F5 again to start another debugging session and repeat the above steps and once you've gotten back past the final step you will see that the breakpoints you set in the last debugging session have been remembered in the new one.

What are all these bloody windows?

I have found that using the debugger was made quite hokey due to the fact that the screen space got crowded out with a lot of superflous stuff. I can't see, at this stage, why you really need the TRACE_WINDOW or HELP_WINDOW to be there in every debugging session. The WATCH_WINDOW and the STACK_WINDOW are the only real useful ones.

So I have edited the debugger.py file to do away with these windows. You can download my edited version here

http://www.workingsoftware.com.au/downloads/debugger.py
NB: All the original code has been more or less left in there, I've just commented the various bits out. You can run a diff on the original to see exactly what the changes are.

Now when you start up a debugging session you'll see the file(s) you're debugging, the WATCH_WINDOW and the STACK_WINDOW all stacked on top each other like you'd get if you did :sp normally.

The fine peeps who developed this debugger have mapped the F1 key so that it resizes the debugger windows. This is cool but I found you had to press F1 three times to get the window to resize the way I wanted it. So I edited debugger.py again to make it just toggle between the equivalent of Ctrl+W= and Ctrl+W_ . So when you're debugging if you are in the find_product.class.php file's window and you press F1 it will blow that window up to take up the majority of the screen. Press it again and it will go back to an even split between find_product.class.php, WATCH_WINDOW and STACK_WINDOW. Go into the WATCH_WINDOW window and press F1 and it will make the WATCH_WINDOW blow up, hit again and back to even split. Same goes for STACK_WINDOW obviously.

Controlling the debugger - keyboard commands

I realise this is probably stating the obvious for a lot of people and that it is also covered in the debugger script documentation but I'll include it anyway for completeness... Also if you're using the aforementioned edited version of debugger.py you will no longer have the HELP_WINDOW in your debugging session so you'll have to remember the commands with your brain... this might help:

F12 caveats

• If you hit F12 and the cursor is positioned on an empty line then it will cause vim to go into INSERT mode. Quite annoying but good to know.
• If you hit F12 and the cursor is in the WATCH_WINDOW or the STACK_WINDOW then it will print "no commands none" and end yourde bugging session. Again annoying but good to know.

A couple of handy commands for managing breakpoints

So you can use :B and :Bp to toggle a break point on/off. But some other useful stuff that's not immediately obvious is:

Five seconds doesn't seem like long enough..

The debugger.py script is hard coded to wait 5 seconds for a connection after you've pressed F5. This didn't seem like long enough to me so I changed it to wait 15 seconds.

This change is in the edited debugger.py which you can download here:

http://www.workingsoftware.com.au/downloads/debugger.py

Just search for 'serv.settimeout(15);' and change 15 to however many seconds you think is reasonable if you want to change it.

So they're a few little things I found handy. If you find more let us know ;-)

Comment Archive

This post originally appeared on an older version of the Working Software website which implemented it's own comment mechanism. This new version of our website/blog uses Disqus for comments (see below) however I wanted to preserve the comments made on the previous post here:

hi just to add another usefull thing :Bp would set up a breakpoint with an expression according to context you might wanna check, http://slackdna.blogspot.com/search/label/vim-dbgpclient

Zeft (2009-03-11)

Usually, you can keep the design of your interface pretty well separated from the logic behind it. One example where this falls down, however, is in the case where the interface uses image inputs.

The image input type allows the use of an image for a button. They're not really all that popular due to accessibility issues, but this is not an ideal world and sometimes you have to work to a design brief that includes image buttons.

So the catch with these is that Browsers send the name attribute of the button to the server differently. In Firefox, for example, you get three $_POST variables in PHP:

• button_name
• button_name_x
• button_name_y

The x and y are the positions of the button on the page and are (were) used in server side image maps. However Internet Explorer will send only the latter two values:

• button_name_x
• button_name_y

So the safest way to check if a particular button was clicked in your PHP script (or other server side scripting language) is to look for the parameter with the _x or _y appended (in .NET this becomes .x or .y):

The use of both _x and _y here is redundant but I thought I'd just include it for the sake of completeness.

The problem I have with this is that it ties the way the form is being processed to the way the form appears. So my workaround for this is to always use a function to check for the presence of button parameters:

That way you can check which button the user clicked and it won't matter if a later design change adds in image buttons.

I recently had to take a string with some delimited values as placeholders and replace them with variables from an array. This is useful for things like sending out email newsletters or SMS messages with people's names or other unique information in each message.

An example of a string with placeholders in it, using the string '%%' as a delimeter is:

Hello %%NAME%%, your %%CAR%% is ready to pickup from %%STORE%%

Now take an array with some key/value pairs:

The following code will perform the replacement based on the filter:

The interesting parts of the above code are, in the pattern:

the "?" makes this a "non-greedy" match, meaning that it won't match everything in between the first and last occurence of $delim. Using "?" in this way allows you to make individual matches non-greedy. You can also use the /U modifier at the end of the expression to make all matches non greedy:

The next interesting bit is the fact that the second argument to preg_replace_callback can be an array with an object and a method. This allows you to use preg_replace_callback in your OO based PHP application without breaking out into procedural code (cos where would you put it!?).

UPDATE August 19, 2008 11:22 GMT+11: This article has been superceded by an update with more permanent URLs: International Mobile Number Validation PT II

Validating mobile phone numbers in international format isn't all that easy. Numbering plans are disparate and constantly changing.

I've come up with a parseable file format which is published here:

http://www.workingsoftware.com.au/mobile_numbering.txt (ed: This link is broken, an updated link is available in the follow up post International Mobile Number Validation PT II)

I've written a class file in PHP (5 only) that reads this file and allows you to validate a number:

http://www.workingsoftware.com.au/mobile_validation.class.php.txt (ed: This link is broken, an updated link is available in the follow up post International Mobile Number Validation PT II)

There's documentation at the top of both those pages I won't bother repeating here.

I think the only way we can make this work is to have it collaboratively maintained. I've proposed that the project be hosted at http://www.kannel.org.

Please feel free to contact me or subscribe to users@kannel.org if you would like to discuss this further. Together, we can validate!

I'm working on a job for a client where legacy database data are being used to generate an XML document for processing with an XSLT stylesheet.

The data are encoded HTML entities in the database. So when I created my DOMDocument, I got the following warnings:

Warning: DOMDocument::loadXML() [function.DOMDocument-loadXML]: Entity 'middot' not defined in Entity, line: 963 in /usr/local/www/data-dist/sheds/includes/SDEHSFunctions.php on line 414

Instead of passing in '·' in the XML string to the constructor of the DOMDocument object, I needed to either declare all entities in the XML doctype (bothersome) or I needed to convert these text entities into numeric ones (eg. '·' becomes '·').

I took a look around and found this handy function:

http://php.net/get_html_translation_table

I did a print_r on the translation table returned and found that it returns an array where the key is the actual character represented and the element is the textual HTML entity. So here's a quick function to get the character coded equivalent:

Introduction

Installing kannel (and many other packages) on Fedora is simplified by using the yum package management system. Unfortunately I couldn't find a repository with an rpm built for Kannel 1.4.1 on FC3 because FC3 is so out of date.

So the following is a little collection of tidbits I found whilst going through the process of installing Kannel 1.4.1 on an FC3 system

This covers how to build the RPM from the DAG src RPM file. As far as I know you should be able to use the RPM I built like this:

If that doesn't work for you, then you may need to check out your yum configuration (see section below) or alternatively you should be able to go through this whole article to build your own src RPM.

DAG Packages

There is a large repository of RPM's for FC3 located here:

http://rpmfind.net/linux/RPM/dag/fedora/3/i386/

The only problem is that there is no RPM available for Kannel 1.4.1 on FC3, only Kannel 1.4.0. Version 1.4.1 of Kannel has many bug fixes so I need to rebuild the RPM from source.

I found the DAG src RPM for Kannel here:

http://dag.wieers.com/rpm/packages/kannel/

Rebulding RPM from source

In order to rebuild the RPM from src, I did the following as root:

This then told me that a bunch of dependencies were missing. I tried adding the DAG repo to my yum configuration but failed, so I couldn't get yum to recognise "kannel" as an argument.

Configuring yum

I looked around and found all the stuff I needed to configure yum properly to look in lots of useful repos, like livena et. al. I've put my yum configuration in a tarball here:

http://www.workingsoftware.com.au/downloads/yum.conf.tar.gz

You should just need to do this as root on your FC3 box to get yum configured (make backups of your /etc/yum.conf and /etc/yum.repos.d prior to running this):

This will create a yum.conf file and a directory called yum.repos.d that will look in lots of useful places for packages.

Resolving dependencies

So now yum is looking in all sorts of useful places for packages, I issue my rpmbuild command again:

It fails, telling you what packages are missing, eg. libxml2-devel. For each of these missing packages, just use yum to install them. There aren't many, so it won't take too long. For example to install the missing libxml2-devel package I just did:

Installing Kannel from the RPM you built

Once you've installed all the missing dependencies, your rpmbuild command will successfully run and give you the location it wrote the RPM to. I looked and found that it had written the RPM to the location:

So I just did:

As an interesting side note, if you ever wanted to use this RPM in future, you could use yum to automatically resolve your dependencies by doing:

This means that yum will install any dependencies which it is configured to look for (if you use the yum config above this will happen).

I've made the RPM available here:

http://www.workingsoftware.com.au/downloads/kannel-1.4.1-2.rf.i386.rpm

Conclusion

Although FC3 is a really old release and we really shouldn't need to be doing all this, it just so happens that in this instance it was more time efficient to stick with the outdated release and figure out how to install Kannel on it rather than upgrading to the latest fedora core.

UPDATE Tuesday December 28 17:59 GMT+11: It's a long time since I wrote this original post but it still gets quite a bit of attention so I've written an updated version Apple is Still Crap.

I recently rented a MacBook Pro in order to develop some dashboard Widgets for a client, and I have to say I'm morbidly disappointed with it. I think Apple should spend less time trying to squeeze more cash out of their consumers and more time creating a product that works.

Here is why Apple is crap:

• OSX System upgrades break: this actually happened a little while ago to me, not on the MacBook Pro that I have currently. I did a system upgrade, and the machine rebooted to a blank, blue screen. I had to spend an hour on the phone to Mac technical support and needed to run some kind of repair script off the installation disc
• OSX is unstable: I've only has this MacBook Pro for 5 days and it's crashed three times. I'm amazed. They took the most stable operating system in the world (BSD) and turned it into a flakey house of cards. It actually crashed during a restart! I told it to restart, and it came up with a message telling me it had crashed and I needed to restart. Compare this to the FreeBSD system I run on my desktop using XFCE4 as the desktop environment which has never crashed. The only time it has ever frozen up on me is when I bumped the casing too hard and it unseated one of the PCI cards. That's in nearly 5 years of operation. It's never ever crashed. The MacBook Pro crashed three times in 5 days. What a piece of garbage.
• iTunes DRM: Apple have really made a coup of this whole digital song sales thing. You buy these songs for $1.69, but you can't play them on more than 5 computers and you can't even burn them to a CD! Bloody useless. You'd think that you'd be able to get more usage out of a product that you actually paid for than one you stole (ie. you can do whatever the hell you want with a file you download illegally. What's the incentive to buy their product?!)
• Firefox runs like a bitch on OSX: I like firefox as a browser but the Mac port runs terribly and Safari, although an okay browser, doesn't have all the really cool features that make web development really easy like the view source chart addon, web developer toolbar and a detailed error console.
• There's no right mouse button: why why why? Why would you not have a right mouse button? The Apple team are obviously aware of the usefulness of a context menu, because you can still get one by doing ctrl+click. Why would you make it so you need two hands to right click instead of just putting another button on the damn mouse/trackpad? What's wrong with these people?
• Apple == Landfill: Apple hardware becomes obsolete really fast and then you can't do anything with it. I've tried installing Ubuntu and NetBSD on old Apples and it's nothing short of suicide inducing. Half the time it just doesn't work. And now Apple has changed to x86, I guess at least now you'll be able to install other OS's once Apple's system crippling OS updates render your computer useless. Not to mention the temporal nature of the iPod and it's relationship with the ever upgrading OSX and iTunes versions. Disgusting
• The package management systems (fink and darwin ports) are the worst of any *nix distro. EVER! Also, even though OSX ships with a unix command line terminal, it doesn't ship with make unless you install the developer tools. Why? Would including make by default really have crippled the system so badly that you just couldn't fit it in?

Quite frankly, I hate Apple and wouldn't buy one of their products if my life depended on it.

Comment Archive

This post originally appeared on an older version of the Working Software website which implemented it's own comment mechanism. This new version of our website/blog uses Disqus for comments (see below) however I wanted to preserve the comments made on the previous post here:

Whatever you think about your mac experience, Windows is by far much worse, and unfortunatley, it's everywhere you look.

sustainedenergy (2007-07-23)

Well, yes naturally it's better than windows :) I'm actually just kind of jaded because it's based on BSD so naturally I expected immense stability and usability out of it as a UNIX distro which, in my (somewhat limited) experience, it hasn't delivered well at all.

Iain Dooley (2007-07-23)

Also... as much as I hate the company, the Dashboard and Expose make my day...

Iain Dooley (2007-07-23)

i think your being a little harsh. crashed 5 times in one day? seriously? i've had my book for amost a year and i don't even thinks its crashed 5 times total. as for the hardware, i had a dell before my book, and i couldn't use the thing for 15 minutes without having to plug it back in. my book on the other hand runs for hours, isn't loud and doesn't get quite as hot as my dell did. maybe you just have a defective book...

taelor (2007-11-11)

you rented a machine that was maybe unstable to start with... I'm running web and e-mail and backup server on a 500 Mhz G4 cube that hasn't crashed 5 times in over 5 years, and it's up all the time !

peter (2007-12-11)

good point about Apple == Landfill. hadn't thought about it before. i've been a PC user / builder all my life and have built most of my computers from boxes that i found on the side of the road - effectively reducing landfill. i too also HATE the one button thing. they've had to invent another 65 buttons to compensate - nice one APPLE you bunch of morons. not to mention the inconsistent function of keys like 'Home' and 'End' across applications. for like 10 years the End key took me to the end of a line. then all of a sudden i jump on a MAC and i'm getting taken to the end of the document in some apps. just plain dumb. do these people think that it's cool to change things that have worked for years as some way of making their product unique? MACs are definitely NOT developers tools. they're more like expensive toys for jerks who think that things that look cool work better. there! i've had my piece

cambit (2008-02-08)

cambit - you have put into words what I have thought for years. Nice one!

Tomass Zutis (2008-05-22)

2.4 QUAD CORE 750GB S-ATAII 4GB 1066Mhzram ATI HD4850 DDR3 22"Widescreen TFT Windows vista X64 Runs everything & if you protect it there's no fear 4 virusses. Price for all this? €900 Price for apple system with slower cpu & crap gpu? €2.250 That leaves me some money to get wireless stuff + a design case and mouse with one button. Right?

Glenn Vercamer (2008-08-21)

i fucking hate apple. i bought my imac 2.4 ghz intel core 2 duo with an upgrade of 2gb ram in mid june, it was the second from the best and most expensive imac available at the approved apple dealer. come middle of august and it starts making strange sounds. apple authorized technicians can't find a fault. send it back. come september, 3 months after purchase, it starts shutting down on me randomly. over 4 times in a day. i was not overworking it. merely on youtube, facebook and probably itunes. i take it to another authorized apple technician, 'oh it probably isn't stable, you need to install your updates.' why should i need to install updates on my computer when it is only 3 months old. shouldn't i be able to buy a top of the line state of the art technology and for it to just work. why do i need to constantly do these updates. i was not made aware of this by the man at the apple shop. that if i didn't then my computer would shut down on me in the middle of important work i was doing. i hate apple. plus. what good is a warranty if they dont know whats wrong and they charge you anyway for an 'inspection fee' when they haven't done anything. one other thing. why are they so keen to sell you the apple protection care plan if they claim their product is so great. why why why

ting (2008-10-07)

"Quite frankly, I hate Apple and wouldn't buy one of their products if my life depended on it." WORD :D Good on Steve for taking all the fools money though, I'd do the same if I could.

Nik (2008-12-01)

You know, I was pretty proud of what I had read until you commented that Windows sucks. I have to say, for a company that is forced into writing an OS that should maintain compatibility with literally BILLIONS of independent applications as well as be more customizable than any other product in the industry, Microsoft does a damn good job. When I write code, the variables can be limited to a certain set of scenarios, when they develop their OS they have to consider limitless scenarios. And for those of you who whine that Windows is riddled with viruses, Apple isn't without fault, it's just that no one wants to bother with writing a virus that affects 3-5% of the total PC market. If you own an Apple and think you're more cool because of it, just shoot your lame coffee shop loving ass already, because I really can't stand people who will pay big money for a laptop with no optical drive.

Dustin (2009-03-11)

Oh Iain, I really suggest you try a Zune...limited monthly cost (restricted to the device if you pay a monthly fee of around 20 US Dollars) and you get 10 absolutely free downloads, DRM free. Plus, if you know your laws, you know you have the right to remove the DRM restrictions and utilize the music on any device you choose through the use of ripping software, so long as you continue to pay your monthly fee.

Dustin (2009-03-11)

I agree Apple is very bad. The media may say it is better than Windows. When I try it, Apple is WORSE than Windows. Poor media, it contains a lot of useless marketing stuffs...

andjohn2000 (2010-03-07)

Great article I totally agree with everything you are saying Apple are too profit focused. I can build myself a top end PC for $700-800AUD and I don't have to really upgrade for perhaps a year and i might just spend $100 on more ram or something, not another $4000 Mac.

Luke (2010-09-17)

why does my iPod erase all my music....why does my iPod not remember my security information...why does Apple require all my security information and then use to block me from getting online help....therefore charging me $30 plus tax to get phone help....i wish I'd never bought this thing....my music is gone....my podcast idea is gone ...totally frustrating.

kathy (2010-11-11)

Here is my little recipe for getting PHP4 and PHP5 to run concurrently on FreeBSD using Apache 2.2 and mod_proxy.

First, check this out:

http://wiki.coggeshall.org/37.html (ed: this link is currently broken - I've emailed John to see if he has an updated link)

That's the basic recipe, however there are a couple of problems with it. The first is what appears to be a syntax error (perhaps an incompatibility between 1.3 and 2.2 versions of Apache) that I fix later, the second is that this ProxyPass only does half the job! I found that, using this method, if i went to the PHP4 host, it displayed the correct index page, however if i put something like http://php4.myhost.com/subdir then it changed the URL of the browser to be http://127.0.0.1:8081/subdir! This wasn't to do at all. Thanks to megapaz in the apache chatroom on freenode for pointing me to this next article (and thankyou to niq also for writing it!):

http://www.apachetutor.org/admin/reverseproxies

So I also need a reverse proxy. That article is really long and deals with the issue in great detail, more detail than I required. I do not need to deal with multimedia content, javascript links or problems with links in general because I am only using PHP4 to run one very specific application.

There is one specific upshot of this: the discussion below deals with compiling the proxy modules from the FreeBSD ports system, however when I tested it out, this did not include all the modules mentioned in the above article, so in order to get all those modules compiled in, you will need to look into which 'KNOBS' you can turn on (see below) or manually change the Makefile.

Compiling with mod_proxy from FreeBSD ports

Because when you install from a port in FreeBSD you don't actually type ./configure, you have these things called 'knobs' that allow you to set/remove common options. In order to configure with mod_proxy when installing Apache 2.2 from the ports, I did this:

I then include the following lines up the top of httpd.conf (where I
have added line breaks for readability, they are marked with "\"):

Installing another Apache instance

For this I just downloaded the latest version of Apache 2.2 from the apache.org website and cut and pasted the instructions in John Coggeshall's tute above to install it without the ports.

PHP Installation

Now that I had Apache 2.2 installed from the ports, I went through my usual PHP5 installation for which I don't actually use the ports. So I just get the latest from php.net and have a script called runconf.sh which has my configure line in it that I've been using forever:

Once that was all up and running, I needed to install PHP4. So I downloaded the latest version of PHP4 from php.net, and used the following line in configure:

So when I installed, my PHP4 installation was located in my second Apache installation folder. I then copied php.ini-recommended into:

And downloaded the latest APC (Alternative PHP Cache). I wanted to install it for the PHP4 installation, so I followed the instructions in the INSTALL file, but used the phpize and php-config included in the php4 folder I had installed under my apache_php4 installation:

If you haven't used phpize on this machine before it will complain it can't find autoconf etc. FreeBSD comes with autotools 253 and 259. I've found that 259 works for phpize, so I do:

Configuring and running Apache

I won't bother going through Apache configuration here, but there is a mistake on the Coggeshall wiki (well, for v2.2 it doesn't work, perhaps it worked for 1.3 or something). I put a file called php4.conf in /usr/local/etc/apache22/Includes/ with the following in it:

And use the 'Listen' directive explained in the Coggeshall wiki in the instance of Apache that is running PHP4.

I then run my Apache PHP5 instance as normal, and use the good old apachectl for the PHP4 instance:

Bug 37770

After running for a few hours, I noticed that the Proxy would mysteriously stop working, and would return a 502 error page with the error:

proxy: error reading status line from remote server 127.0.0.1

This was a real doozy, as it was a very intermittent bug and didn't seem to be related to load or anything in the code that was being executed. I tried running httperf on it to load test and could not reproduce the error reliably. After some reading, I found this blog post mentioning a similar error using mod_proxy to run Mongrel for Ruby on Rails hosting:

http://blog.pomozov.info/posts/apache-mongrel-and-proxy-error.html (ed: This link is broken. I've attempted to contact Anatol Pomazov to see if there is an archive of the content somewhere)

I also found this mentioned here:

http://httpd.apache.org/docs/2.0/mod/mod_proxy.html#envsettings

So from this I concluded that my VirtualHost include should now look like this:

I then restarted both the PHP5 and PHP4 Apache instances:

and I am yet to see the error recur having run the system for nearly 48 hours since doing it. If the error does recur I'll update my post but it appears to be fixed.

That's it! Now I can point my browser to http://php4.myhost.com/ and it serves up pages using PHP4, but maintains the correct domain in the browsers URL bar. Because I've used a VirtualHost directive to pass to the PHP4 instance, I can easily make any domain I'd like get served up with PHP4.

UPDATE Wednesday December 29 14:27 GMT+11: I've added another post that follows this up with a fix for the blocking behaviour of PHP sockets: Interprocess Communication in PHP with Non-Blocking Sockets which is intended to complement but not replace this original post.

I recently wrote a little application that dumps a file across a forwarded port. It was tricky, but very convenient because you can do things like report status of the upload process (I could also have done this by parsing the output of something like scp, but I kind of liked having direct access to the copy process/stats).

When I first wrote it, I didn't know what I was doing and had never written socket code before. It was a big procedural mess. Naturally I was keen to separate out my socket class into it's own package but this presented a problem: the controlling process needed to check the status but how could I decouple the process that instantiated the socket class from the socket code itself? I didn't want to hard code the status reporting into the socket code and I wasn't too hot on the idea of passing in some kind of status reporting 'callback object' to receive messages during the process.

I figured what I really needed was a new thread. I'm not sure if that's exactly what a fork is, but it's close enough and serves my purposes. You can fork in PHP using the pcntl_fork method:

http://php.net/pcntl_fork

When I first started out, I thought that this would be really easy. Think again fool! It did turn out to be quite easy, but just not obvious. For starters, I read this article on PHP forking (ed: the original article was located at http://www.van-steenbeek.net/?q=php_pcntl_fork which has been taken offline, although the author was good enough to organise an archive) and the man page at php.net.

As you can see from the PHP manual, in order to use the features in this article you will need to compile PHP with the following options:

OO and Me

I'm a nut on object orientation. To me it makes loads of sense so I always write stuff with objects. The rest of this article uses objects but the issues I'm discussing apply equally well in a procedural arena.

The First Barrier - Copy On Write

When you fork a process, the script splits in two (like Lorraine in that episode of Astro Boy when she gets into the robot fighting league). As "Jeff" on the pcntl_fork man page describes (in the user contributed notes), this operation is not expensive because it uses a "copy-on-write" model. This means that, until each process writes to a variable, that variable is shared between the two processes. Great! Inexpensive.

But this causes some problems. Consider the following piece of code (which was my first attempt):

If you run that code with PHP on the command line, it will swamp your screen with 0's. Oh if only it were that easy!!

See the problem is that as soon as the child process writes to the $this->up_to variable, it gets it's own copy. So when the parent process checks if $this->up_to < 1000, it's always checking it's own copy, which is always 0 and never changes.

Interprocess Communication using Sockets

So it turns out there is a really convenient way to communicate between processes using the PHP socket_create_pair() function!

You can read about it here:

http://php.net/socket_create_pair

There's even an example on there doing exactly what I was after! So after reading that I was sold. The only thing left to do was to put it into a nice, reusable object pattern. The code I came up with is below, and I'll explain each part of it for you in detail:

What??

Okay let's take a look at the various parts of that code.

So this is pretty standard. I'm defining a class called Threader. Interesting to note that, for reasons you'll see later on, I've chosen to implement this class using the Singleton pattern, so I've got:

The other thing to notice is that the constructor is private. That's because in a Singleton pattern you never instantiate a class directly. You do it with a factory method like this:

This means that when you call:

you will always get the same instance of the Threader object that is instantiated the first time you call it. Of course this does not hold true in the case of forked processes (I had hoped that it would but I wasn't so lucky). Now we get down to the nitty gritty.

The countToOneMillion() method is the one that we want to fork because it's going to take so long and we want to have status updates. So the first thing to do is create the sockets we'll use for interprocess communication:

This is straight off the socket_create_pair man page. The only tricky bit here is that we are taking the second of the two sockets we created and storing it in a member variable. Now comes the fork we've all been waiting for:

I won't got too much into explaining the fork code because the PHP manual pages as well as the article I posted above do a good job of that. I'll just draw your attention to two things: firstly when I fork, I assign the $pid returned to a member variable of the Threader instance. Luckily for me, this adds it to both the Threader instance in the child process and the parent instance.

This means that where the code checks if(!$this->pid) that code is only executed in the child process. This is where this example differs significantly from other examples in the manual and tutorial above. Because we set the $pid as a member variable, we are now free to go about our merry way. As the child process counts up to one million, it reports its status by writing to the first of the pair of sockets we created.

Curious Interlude

Although you can't write to variables owned by other processes, all the resources are shared. This means that file descriptors, sockets and database connections are shared between the two processes. This is why we can share sockets between the two processes. There is an interesting article explaining this here:

http://www.hudzilla.org/phpbook/read.php/16_1_4

Back Into It ...

So now we have a child process which is going to happily sit there and count to one million, and keep us up to do date by writing status messages (in a pre-determined format) to a socket. So we just have to listen to the other end of the socket! Here is the code that will do that for us:

The first thing it does is check that $this->pid evaluates to true with if($this->pid). As you'll recall, because we assigned the $pid returned from pcntl_fork to a member variable of the Threader instance, we can now use that to check if we are in the parent or child process at any time. As it states in the PHP manual, a $pid > 0 means we are in the parent process. Convenient!!

We then call the socket_read function to get the data that was written using the socket_write function from our other process. One not so convenient part of this is that you have to provide a length for the messages, which I've included as a member constant of the Threader class. What this means is that, when the child process writes to the socket, we will only get one message at a time, and we won't miss any messages.

I then just use a few simple if/else statements to check what the message was. If it was an "UP TO" message, which means that it was updating the status of the progress in counting up to one million, then I set the $up_to member variable on the object. Now ...

The Code that Calls It

This is what I've been building up to the whole time, and chiefly why I just wasted a day doing this when I could have just as easily passed a callback object. Because I've used a Singleton pattern and because the pid is stored in that Singleton, the code inside the Threader can be completely decoupled from the code that calls it. So:

First thing to note here is that I'm accessing the Threader class using it's current() method rather than instantiating one using the 'new' keyword. As I mentioned above, this is because the Threader class implements a Singleton pattern. So when we check Threader::current()->pid(), we ensure that the code will only execute in the parent process. The child process is busy ticking away the whole time since we called countToOneMillion.

The real magic happens when we call:

If you'll remember, that's where we read the status messages from the socket that are being written by the child process. This method also updates the $up_to variable in the Threader instance. We can then echo out what number we are up to.

Now something screwy happened here. I didn't want to echo out a million numbers, so I put a sleep(30) call in so that the status would only update once every 30 seconds. However it just hung. Never went anywhere. If anyone knows why I'd love to hear it. So instead I just put in some code that only echo's out the number it's up to if it's more than 1000 greater than the previous update.

Conclusion

The actual result above is identical to what could be achieved by simply echo'ing out the values in the countToOneMillion method. However the significant thing is that you can have a class which performs some time consuming task, then in the meantime you can do something else like send someone an email or SMS updating the progress.

The other significant thing is that the class performing the time consuming function doesn't need to know anything about the environment in which it is being called. In terms of creating a reusable class that you can drop into any project this is highly desirable.

Happy forking!

Comments Archive

This post originally appeared on an older version of the Working Software website which implemented it's own comment mechanism. This new version of our website/blog uses Disqus for comments (see below) however I wanted to preserve the comments made on the previous post here:

For the 30 sec issue: I would bet that PHP simple timed out. See here: http://www.php.net/set_time_limit

Peter Nagy (2008-06-01)

OOPS!! Don't know how that got in there :) fixed now. thanks for stopping by!

Iain Dooley (2007-06-11)

Hi. Thanks for the article with the link for my site. I'm sorry to see you've got the link wrong, though :-) Your link is: http://www.van-steenbeek.net/%EE%9B%8Fq=php_pcntl_fork And it should read: http://www.van-steenbeek.net/?q=php_pcntl_fork Again, thanks. Keep up the good work!

FST777 (2007-06-11)

Wow, good job, that's some really interesting code, thanks for publishing.

dave (2007-06-08)

good to see you got it working, and even better that you wrote about it :-D - ecoleman

Eric Coleman (2007-06-08)

UPDATE Tuesday December 28, 2010 05:27 GMT+11: I have combined this post and the post it followed up into one more coherent, useful post: Installing an Independently Verified SSL Certificate. This post is preserved in it's original form for historical purposes. The original article this was in follow up to is: Creating a CSR in FreeBSD.

I recently wrote about generating a Certificate Signing Request (CSR) using openssl on FreeBSD.

So I've now received my signed certificate back from Chris Langlands at www.backend.com. It's time to install it! Luckily this is dead easy. All you have to do is take the certificate that's been sent to you, which will look something like this, included in the email from your certificate provider:

-----BEGIN CERTIFICATE-----
MIIDUTCCArqgAwIBAgIDBfUGMA0GCSqGSIb3DQE
MRwwGgYDVQQKExNFcXVpZmF4IFNlY3VyZSBJbmM
IFNlY3VyZSBHbG9iYWwgZUJ1c2luZXNzIENBLTE
MDgwNTA3MDIzOTUxWjCBwDELMAkGA1UEBhMCQVU
bnN3Lmdvdi5hdTETMBEGA1UECxMKR1QzODI4NzY
dy5nZW90cnVzdC5jb20vcmVzb3VyY2VzL2NwcyA
YWluIENvbnRyb2wgVmFsaWRhdGVkIC0gUXVpY2t
dy52cGIubnN3Lmdvdi5hdTCBnzANBgkqhkiG9w0
zRDf64/CyTTzG+y2OIMTM2p89MjtGndZPn8FtS6
mNSz7P6LcTWf6ihvQ7fA5bL/nilw3Oc+Aqsl+ts
uTrxpsX50xctEKSSC/29ofwAwPJBwgaaRP2x3j8
Af8EBAMCBPAwHQYDVR0OBBYEFHpx92ZbXkTFv/r
MDIwMKAuoCyGKmh0dHA6Ly9jcmwuZ2VvdHJ1c3Q
LmNybDAfBgNVHSMEGDAWgBS+qKB0clBrRLfJI9j
BggrBgEFBQcDAQYIKwYBBQUHAwIwDAYDVR0TAQH
AAOBgQAP3oDeE6cK/M2CnAqDAZJIQGdIesBSIdQ
wa7biEylNXIkh4PicCG8ko8S/zGq3qcGElLLpPG
+BIF4vXMDFBsoss8CVGsZsW67Qw2QMLPp66pYyW
-----END CERTIFICATE-----

and save it into a text file on the web server. When I created my CSR I saved my private key file as vpb-key.pem, so I copied that certificate into a file called vpb-cert.pem and saved them as:

/etc/ssl/key/vpb-key.pem
/etc/ssl/crt/vpb-cert.pem

and then added the relevant lines to my Apache httpd.conf:

SSLEngine on
SSLCertificateFile /etc/ssl/crt/vpb-cert.pem
SSLCertificateKeyFile /etc/ssl/key/vpb-key.pem

For more details on configuring Apache to work with SSL, check out this article on FreeBSDMadeEasy.com.