mwop.net :: Blog Posts

PHP decoding of Javascript encodeURIComponent values

Recently, I was having some issues with a site that was attempting to use UTF-8 in order to support multiple languages. Basically, you could enter UTF-8 characters — for instance, characters with umlauts — but they weren't going through to the web services or database correctly. After more debugging, I discovered that when I turned off javascript on the site, and used the degradable interface to submit the form via plain old HTTP, everything worked fine — which meant the issue was with how we were sending the data via XHR.

We were using Prototype, and in particular, POSTing data back to our site — which meant that the UI designer was using Form.serialize() to encode the data for transmission. This in turn uses the javascript function encodeURIComponent() to do its dirty work.

I tried a ton of things in PHP to decode this to UTF-8, before stumbling on a solution written in Perl. Basically, the solution uses a regular expression to grab urlencoded hex values out of a string, and then does a double conversion on the value, first to decimal and then to a character. The PHP version looks like this:

$value = preg_replace('/%([0-9a-f]{2})/ie', \"chr(hexdec('\1'))\", $value);

We have a method in our code to detect if the incoming request is via XHR. In that logic, once XHR is detected, I then pass $_POST through the following function:

function utf8Urldecode($value)
{
    if (is_array($value)) {
        foreach ($key => $val) {
            $value[$key] = utf8Urldecode($val);
        }
    } else {
        $value = preg_replace('/%([0-9a-f]{2})/ie', 'chr(hexdec($1))', (string) $value);
    }

    return $value;
}

This casts all UTF-8 urlencoded values in the $_POST array back to UTF-8, and from there we can continue processing as normal.

Man, but I can't wait until PHP 6 comes out and fixes these unicode issues…

Overloading arrays in PHP 5.2.0

Update: I ran into issues with the ArrayObject solution, as there was a bug in PHP 5.2.0 (now fixed) with its interaction with empty() and isset() when used with the ARRAY_AS_PROPS flag. I tried a number of fixes, but eventually my friend Mike pointed out something I'd missed: as of PHP 5.1, setting undefined public properties no longer raises an E_STRICT notice. Knowing this, you can now do the following without raising any errors:

class Foo
{
    public function __set($key, $value)
    {
        $this->$key = $value;
    }
}

$foo        = new Foo();
$foo->bar   = array();
$foo->bar[] = 42;

This is a much simpler solution, performs better, and solves all the issues I was presented. Thanks, Mike!

svn:externals

I was recently working with someone who was using Zend Framework in their project. To keep things stable and releasable, he was doing an export of framework into his repository and checking it in. Since files change so much in the ZF project currently, instead of doing an rsync from a checkout into his own repository, he decided instead to delete the directory from the repository and re-add it everytime he was updating framework.

This seemed really inefficient to me, especially considering that it made it incredibly difficult to merge changes from his development branch into his production branch (deleting and re-adding directories breaks the merge process considerably). I knew there had to be a better way.

I'd heard of the svn:externals property before, but never really played with it. As it turns out, it exists for just this very type of situation. The problem is that the documentation of svn:externals in the SVN book doesn't indicate at all how the property should be set, and most howto's I've read omit one or more very important details. I finally figured things out through some trial and error of my own, so I'm going to share the process so others hopefully can learn from the experience as well.

It's actually pretty easy. This assumes that your project layout looks something like this:

project/
    branch/
        production/
    tag/
    trunk/

In the top of your project trunk, execute the following:
```
$ svn propedit svn:externals .
```
This will open an editor session. In the file opened by your editor, each line indicates a different external svn repo to pull. The first segment of the line is the directory where you want the pull to exist. The last segment is the svn repo URL to pull. You can have an optional middle argument indicating the revision to use. Some examples:
- Pull framework repo from head:
```
framework http://framework.zend.com/svn/framework/trunk
```
  - Pull framework repo from revision 2616:
```
framework -r2616 http://framework.zend.com/svn/framework/trunk
```
After saving and exiting, update the repo:
```
$ svn up
```
Commit changes:
```
$ svn commit
```

One thing to note: any directory you specify for an svn:externals checkout should not already exist in your repository. If it does, you will get an error like the following:

svn: Working copy 'sharedproject' locked
svn: run 'svn cleanup' to remove locks

I show using revisions above; you could also pin to tags by simply checkout the external repository from a given tag. Either way works well.

Then, when moving from one branch to another, or from the trunk to a branch, you simply set a different svn:externals for each branch. For instance, your current production might check from one particular revision, but your trunk might simply track head; you then simply determine what the current revision being used is on your trunk, and update svn:externals in your production branch when you're ready to push changes in.

Hope this helps some of you out there!

Getting XDebug working with PHPUnit for code coverage reports

I've been playing with PHPUnit a lot of late, particularly with framework development. One thing that's always hard to determine is how well your code is exercised — basically, how much of the code is tested in the unit tests?

In PHPUnit 3, you can now generate code coverage reports using XDebug, and the usage is very simple:

matthew@localhost:~/dev/zend/framework-svn/tests$ phpunit --report ~/tmp/report AllTests

The above command creates a coverage report directory report under my tmp directory. You can then browse through the reports in a web browser and visually see which lines of code were executed during tests, and which were not, as well as a synopsis showing the percentage of coverage for any given file or directory — useful stuff indeed!

So, what's the problem? Getting XDebug running.

The executive summary:

Enable the extension using zend_extension = /full/path/to/xdebug.so, not as extension = xdebug.so, in your php.ini
Use the setting xdebug.default_enable = Off in your php.ini.
If compiling using pecl or pear, make sure it compiles against the correct PHP; if not, hand compile it using:
```
$ /path/to/phpize
$ ./configure --with-php-config=/path/to/php-config
$ make
$ make install
```

For the detailed narrative, read on.

MVC changes in Zend Framework

Several months ago, Andi asked me to take the role of lead developer on a refactoring of the Zend Framework MVC components. I agreed, though somewhat reluctantly; I already maintain another MVC library, and wasn't sure how well I could fill the shoes of people like my friends Mike, who had done the initial development on the controller classes, and Paul, who provided Zend_View.

The experience has been incredibly rewarding, however, and I've had the chance to pick the brains of and work with some top-notch developers in the process. In the next week or so, we'll be releasing version 0.6.0 of the framework, and it will include much of my work in the MVC components as part of the core distribution. A big thanks to all those who have contributed opinions, design help, code, tests, and documentation; another thank you goes to Andi for trusting and supporting me in this endeavor.

So, what are the changes? Read on to find out…

PHP Best Practices

Yesterday, Mike and I presented our session "Best Practices of PHP Development" at this year's Zend Conference. It was a marathon three hour tutorial first thing in the morning, and we had an incredible turnout, with some fairly enthusiastic people in the audience.

Download the slides slides for PHP Best Practices.

ZendCon: Best Practices Session

As Mike already noted, he and I are presenting a session on "Best Practices of PHP Development" at this year's Zend Conference and Expo. It has been my fortunate experience to work with Mike in the past, and, as he noted, we had so much fun presenting during last year's conference, we thought we'd do it again.

The session is a pre-conference tutorial session, running for 3 hours on Monday morning, 30 October 2006. Currently, we're shaping up the session into the following subject areas:

Programming Practices
- Coding Standards
- Test Driven Development and Unit Testing
- Project Documentation
Tools and Processes
- Software Configuration Management (SCM)
- Collaboration tips and tools
- Deployment

Emphasis is going to be on working in teams, particularly those operating in geographically diverse areas. With roughly 30 minute blocks per topic, we've certainly got plenty to cover!

If you're coming to the conference, we look forward to seeing you in our session!

Zend_XmlRpc_Server

As noted previously by myself and Davey, I've been working on Zend_XmlRpc_Server for some months now. In the past couple weeks, I've refactored it to push the class/function reflection into Zend_Server_Reflection, and, in doing so, noted that there were further areas for refactoring into additional helper classes. Currently, it now has classes for the Request, Response, and Faults, and all actual XML wrangling is done in those, making the server basically XML-agnostic.

PHP 5's Reflection API

As Davey Shafik noted recently, he and I have been working together on some web services for the Zend Framework. In doing so, I've become very familiar with PHP 5's Reflection API, and am coming to love it.

When I first read about the Reflection API in a pre-PHP 5 changelog, my initial reaction was, "who cares?" I simply failed to see how it was a useful addition to the language. Having done some projects recently that needed to know something about the classes they are using, I now understand when and how it can be used. It shines when you need to work with classes that may not be defined when you write your code — any code that dispatches to other classes, basically.

coLinux Recovery

As I've written previously, I use coLinux in order to have a Linux virtual machine running on my Windows XP install. It runs Debian unstable (SID), which gives me all the apt-get love I could want.

Except when an apt-get based install goes bad, that is, like it did Saturday evening. This is the tale of how I got it back up and running.