PHPUnit via phar with suhosin

The PHPUnit developer has announced the end of life for the PEAR installation method for PHPUnit. So I tried the phar method, but phpunit (and phar) would fail silently. After a little searching, I found a useful post* suggesting that suhosin might be the problem (I only have PHPUnit on my development platform, but I also have suhosin there, so that it more-or-less matches production). That post's suggestion of adding suhosin.executor.include.whitelist=phar to php.ini did the trick.

*That post is for an entirely different package, but it was the same problem I was having.

setting the session.cookie_path in PHP (redirection loop)

This is a quick note about a bug in some PHP code I was working on the other day. It took me a while to figure it out. I was developing and initially testing in google chrome, which is evidently forgiving of this kind of error. Maybe writing this will help someone (and maybe it'll help me not to make the same mistake again).

This was for an application which requires authentication. The controller sends the browser a redirect (to the login URL) if the user is not authenticated. I had set the cookie path with something like the following:

$baseUrl = 'https://www.example.com/gakkk/';

// several lines later...

ini_set('session.cookie_path', $baseUrl);

This worked OK in chrome, but Firefox and MSIE both got locked up in redirection loops. After scratching my head for a while, I finally figured out that I should be doing this:

ini_set('session.cookie_path', '/gakkk/');

The cookie_path (it has that name for a reason) should exclude the protocol, hostname, and port. Information security auditors like to complain about web applications that don't set the cookie path.

Zendcon 2009

Got home last night from Zendcon 2009. Good conference.

Here are a few of the main things I'm taking away from it:

• I'm a fool not to be using APC (and maybe memcache)
• Zendconners really like twitter (I got sucked in: @carl_welch)
• I need to learn more about dependency injection and better OOP methods
• I need to give git a try (had a good visit with the guy at the github booth)

Update (Monday 26 October): I took some pictures, and I've posted them to flickr.

xampp

At times I've wanted to try doing some development work on my eeepc, but the default distribution doesn't come with a LAMP stack (and so far I've been too chicken to try installing something else).

Today I tried installing xampp, and that seems to work pretty well. So far my only complaint is that it doesn't seem to come with any version control tools (like svn), and I don't see an easy way to add/compile them (the eeepc doesn't have gcc).

Thursday at ZendCon

Today is the final day of ZendCon, and it's a half day. It started with sessions, rather than a keynote (the keynote is at the end). I was having trouble figuring out which session to attend, but then I looked at the presenter names, and that made it easy to pick "Scaling Mozilla's websites with PHP" with Laura Thomson. I've heard her speak at OSCON, and she always gives a good talk. She didn't disappoint today, as she told us about how she helped Mozilla get ready for the Firefox 3 "Download Day" (or, as she called it, D-Day). It was a fascinating case study of query/code optimizations, caching solutions, MySQL replication tricks, and other goodness. A couple of fun facts are that mozilla.com uses drupal, and D-Day saw saw 14Gb/s of downloads and 2Gb/s of web traffic.

Next Stefan Priebsch gave a good discussion and demonstration of Selenium. Several people have talked about Selenium at this conference, and it really looks like a great resource.

The closing keynote is by David J. Neff from the American Cancer Society. He told us about sharinghope.tv, a site with user-generated content from people affected by cancer. Pretty cool.

Here's a picture from the closing keynote. That's Cal Evans (ZendCon program chair) on the stage, and the (backs of the) heads of Laura Thomson and Paul Reinheimer. Sebastian Bergmann was also milling about (which would have put my three favorite speakers in the shot), but he wouldn't hold still long enough.

Wednesday at ZendCon

ZendCon started with a really interesting keynote today (I'm usually not big on keynotes). The first speaker was a Zend Framework developer named Wil Sinclair, and he talked about how PHP applications have evolved over time. He introduced the CEO and CTO of Varien, and then talked about their company's development of Magento. Magento looks really cool. It's an open-source e-commerce built on Zend Framework. It appears to be as full-featured as big commercial e-commerce sites like amazon.com. They apparently wrote the whole thing in a matter of months. Impressive. I'll definitely try it if I ever need to make an online store.

Then I checked out "Architecting for PHP5..." with Elizabeth Smith. She covered some of the features new to PHP5 and again encouraged us to use extensions whenever possible ("C is faster than PHP"). Looks like I need to read more about the SPL and the filter extension.

Stefan Esser gave a really good talk called "Lesser Known Security Problems in PHP Applications." Some of it was kind of scary. We even got to hear him announce a 0-day vulnerability with the ZipArchive extension (along with a more obscure problem with HTTP response splitting affecting users of older Netscape proxies).

After lunch was another keynote: "The State of AJAX" with Ben Galbraith. Not a lot of technical information, so not too interesting to me. It was mostly a survey of lots of shiny-looking, bleeding-edge stuff (which may or may not be around in six months). He speculated that HTML5 and (google) gears will be important to the future of AJAX. I was hoping he'd tell me which AJAX framework to use, and he did address this point. He said that it's sort of a toss-up between Dojo, Prototype/script.aculo.us, and jQuery, because they're all small, powerful, extensible (with community support) frameworks. So pick one. *sigh* Thanks.

In his "Phar Scape" talk, Marcus Boerger showed us some neat tricks to do with phar, a deployment tool modeled after Java's JAR.

I went to another talk by Sebastian Bergmann, this one called "phpUnderControl: A Quick Start to Continuous Integration." I'm pretty much the only PHP programmer where I work, so I don't really have to integrate my work with a team. But phpUnderControl looks cool enough that I'd like to try it. Besides, it makes nice pictures that could distract managers. (And Sebastian says that he likes doxygen more than phpDocumentor, so I'll give that a try, too.)

I finished out the day with an UnCon session called "Subversion Tips and Tricks" with Lorna Mitchell and Matthew Weier O'Phinney. It turned out to be mostly introductory in nature, but I still picked up a few things. It was very informal, and it was fun.

Tuesday at ZendCon

Kind of a slow morning at ZendCon (to me, anyway). Started off with the keynote by Harold Goldberg, Zend Technologies CEO. Mostly business-speak, so not too exciting (I'm a nuts-and-bolts sort, I guess).

Then I went to "Of Haystacks and Needles" by Derrick Rethans. This looked interesting to me because of my php|arch article. One of the people who wrote about my article seemed to advocate storing records as documents, like maybe XML documents. I think that's a pretty interesting idea, because storing it like that sidesteps the whole issue of different records having different sets of fields. And I could just store the XML string in a text column or something. But I didn't know how I'd do searches--if I want to find a record submitted by someone whose last_name is Smith, it seems like I'd have to run a DB query to get the XML string for every record, pull it out of the DB and into application memory, parse each XML string, and look for any that match. So I was hoping that this would give me some ideas of how to do this (although lately I've been thinking about stored procedures). Anyway, the talk gave me several things to look at more closely later:

• Apache Lucene


• Zend Lucene (PHP port of Adobe Lucene, lacking some of the former's features)


• Apache Solr (an extension of Apache Lucene w/ a Web services API)


• marjory (like Solr but written in PHP, supports other backends like sphinx and xapian)

Lunch was a catered affair again, but no celebrities this time (not that I realized, anyway).

Things picked up after lunch, starting with "PECL Picks..." with Elizabeth Smith. This was a whirlwind survey of some PECL projects that Elizabeth deems "cool." Here are some of the ones that look interesting to me:

• uploadprogress (useful in AJAX applications)


• Haru (for PDF manipulation)


• SVN


• ssh2


• sphinx and clucene

Next was "Knight Rider Methodology to Software Develoment" in which Eli White covered a lot of development tools and techniques. The theme was how Michael Knight got lots done on (the 80s TV show) "Knight Rider" by using the tools at his disposal (well, tool: the car, KITT). The talk was punctuated by clips from the show (one of my favorites as a kid, and it's far cheesier than I remembered--wow). Here are some fun quotes from Eli's talk:

• "hardware is cheaper than people"


• "use someone else's time as your own" (use libraries rather than re-inventing the wheel)


• "nothing is right the first time: even 'hello, world!' needs internationalization" (debuggers)

After his talk, I'm looking forward to trying out some subversion GUIs like subcommander and rapidsvn.

Paul Reinheimer gave a great talk about how Web 2.0 breaks the browser's Back button, and he talked about a really interesting solution using the yahoo user interface (YUI). He also talked about how to deal with AJAX responses coming back in an order different from that in which they were sent. I think that'll take a few reads for it to sink in for me, but I'm looking forward to trying it out. Paul does PHP training, and he's a really good speaker.

Finally, Eddo Rotman gave a good presentation about PHP errors and exceptions. His talk has inspired me to write a cron job which knows a list of PHP application error logs and emails me the entries made in the last 24 hours: maybe it'll append a marker to each log file and email the entries written since the previous marker.

Monday at ZendCon

Today at ZendCon I attended two tutorials, and I enjoyed them both. In both cases, there was a lot of review, but I also learned new things in both. The morning session was "PHP Developer Best Practices" with Matthew Weier O'Phinney and Mike Naberenzy. They covered a pretty broad set of topics:

• They spent quite a bit of time on source control. They talked mostly about subversion, but they also discussed some git's advantages, particuarly for offline development (like on a laptop on a plane). Some of the other things that were interesting to me were the ability to link one SVN repository to another with the svn:internals feature, using post-commit hooks to send emails (to other developers) or to rebuild documentation (e.g., via phpdoc), and using branches for maintaining older product lines (the example that occurred to me is that the Apache developers might have a branch for the 1.3.x line).


• They talked a lot about coding standards, and then advocated the PEAR standard. I think that's the default standard which is enforced by PHP_CodeSniffer, which is one of the coding tools they discussed (which I started using a few weeks ago).


• They spent some time on testing, and they pointed out that when coupled with xdebug, PHPUnit can do coverage analysis. I didn't know that, and I think that's pretty cool. I've never been able to get xdebug to work for me, but that's a good reason to try again. They also made PHPUndercontrol look interesting enough to play with.


• They talked about phpDocumentor, which is something I've been using and enjoying lately. They also touched on DocBook, which evidently is just fairly straightforward XML. I'll have to give that a try.


• They concluded with a few remarks about deployment issues. I'd hoped they'd talk about phing or phar, and they might have, but they sort of ran out of time.

Lunch was catered box lunches, served in a large room with lots of big tables. So I had lunch with strangers, but one of them turned out to be Jay Pipes. I'd heard him speak at OSCON previously, and he had some interesting insights about Sun's MySQL acquisition (he said that Sun had so far mostly left MySQL alone). Jay was wearing a cool T-shirt with the Decepticon logo. And Paul Reinheimer was sitting behind me. I don't know his face, but I recognized his voice. Sounded like he was keeping his tablemates well entertained.

The afternoon session was "Quality Assurance in PHP Projects" with Sebastian Bergmann. I've been fiddling with PHPUnit for a little while, and the talk gave me several things I want to research a bit more when I have time:

• the --testdox option


• the @group phpdoc tag


• xUnit Test Patterns, a language-agnostic book about software testing


• the DbUnit feature


• using sqlite for testing DB stuff


• using the dataProvider feature for throwing a large stack of specific test data at the code

Sebastian also talked a lot about mocks and stubs and such, but that's still a bit over my head. But we got to see the --ansi feature of the shiny new v3.3.0 (which he just released this morning).

The eee pc had good moments and bad. This convention center did a pretty poor job of providing power outlets (people were sitting up front so that they could plug into surge suppressors for the sound systems, and others even pried some metal plates off the floor to get at electrical outlets), so I was on battery all morning (I scored an outlet in the afternoon). The eee pc battery held, but perhaps only because I dimmed the screen and turned off the wireless. But I'm still happy with it--it's light enough that I don't feel tired or sore at the end of the day. And although I make more mistakes than usual on the little keyboard, it's plenty good enough for note-taking.

LAMP pconnect

I had a bad experience a few years ago using pconnect() in a PHP program running in/on/under Apache on Linux to talk to a MySQL database. My memory is somewhat vague, but we had to restart the MySQL service (and switch to non-persistent connections). Ever since then, I've had an irrational fear of pconnect() and have never used it.

Looks like I'm not the only one who doesn't trust pconnect().

PHP frameworks: performance

For some time now I've been trying to figure out what to do about PHP frameworks. Most notably I've been wondering which one would be best for my needs and whether or not it's worth my time learning one. The problem is that there are so many. Zend, Cake, and Symfony appear to be pretty popular (at least, they are frequently mentioned in PHP blogs).

developertutorials.com has a post which highlights a recent performance analysis of Zend, Cake, and CogeIgniter. The concluding sentence is a good summation:

...if you’re building small applications, CakePHP will clearly save you time in development, CodeIgniter will offer massive performance benefits, and Zend will give you a reasonable middle ground.

Still not sure which one I want to try, but I lean a bit toward Zend (although it has received some recent criticism regarding its coding conventions and documentation). I like it that you can use it as a full-blown framework or just import the features you need for your application.

When I get some time, I guess *shrug*

mcrypt randomness in PHP

The other day I was trying to debug a PHP program which was being really slow. I'm not bright enough to use proper debugging tools (like xdebug), so I just sprinkled in a bucketload of error_log() calls until I figured out what was gumming up the works.

It turned out to be an mcrypt_create_iv() call. Sometimes that call would run in a fraction of a second, and sometimes it would take over a minute, with no discernible pattern. I had pretty much done a copy-and-paste from the mcrypt_module_open() manual page, which shows using the MCRYPT_DEV_RANDOM constant as the second argument to mcrypt_create_iv(). It finally occurred to me to try using the MCRYPT_DEV_URANDOM constant (note the "U"), instead, and the encryptions immediately became consistently fast.

This was happening on a VMWare ESX guest running RHEL4. A Web search or two found a good comparison of /dev/random v. /dev/urandom, and my problem turned out to be a good illustration of /dev/random blocking the caller until sufficient entropy is attained.

AdoDB, PHP, MySQL, SSL

(I'm practicing for an "Unreadable blog post title" contest.)

Here are a few hints on how to use SSL certificates when connecting to MySQL from a PHP program using the AdoDB database abstraction layer. (You may want to see my previous post on setting up SSL certificates for MySQL connections.)

The trick is to use a DSN in the NewADOConnection() call (rather than authenticating with a Connect() call) and to use the mysqli driver (looks like the mysql driver won't work for this). The DSN syntax allows you to supply client flags, and there's a mysqli flag for using SSL certificates.

After creating a CA certificate (we'll say it's at /path/to/ca-cert.pem), make sure that the following item is in the [client] stanza of /etc/my.cnf or the connecting user's ~/.my.cnf on the client host:

ssl-ca=/path/to/ca-cert.pem

Then try the following PHP program:

// these are part of the AdoDB library
require '/path/to/adodb-exceptions.inc.php';
require '/path/to/adodb.inc.php';

/*
* I got the '2048' from running
* printf( "%d\n", MYSQLI_CLIENT_SSL )
* in a PHP program (w/ the mysqli extention installed)
*/
$dsn = 'mysqli://ssluser:sslpass@dbhost/test?clientflags=2048';

$dbh = NewADOConnection($dsn);

$sql = "show status like 'ssl_cipher'";
$res =& $dbh->Execute($sql);
print_r( $res->fields );
$res->Close();
$dbh->Close();

This should generate output similar to like this:

Array
(
  [0] => Ssl_cipher
  [Variable_name] => Ssl_cipher
  [1] => DHE-RSA-AES256-SHA
  [Value] => DHE-RSA-AES256-SHA
)

problem w/ PHPUnit reports on CentOS5/RHEL5

Yesterday I was trying PHPDocumentor and was going through its Quickstart guide. After I ran phpdoc on the sample code, I threw the reports in my CentOS5 Apache document root so that I could look at the output. Several of the pages wouldn't load. After a quick look at the Apache error log, I saw that those pages were generating PHP errors (the T_STRING gripe), even though the files were named something like sample.php.html.

(CentOS5 and RHEL5 have Apace v2.2.x.)

It took me a while to figure it out, but it's due to an odd feature of Apache which honors multiple extensions in filename. RHEL5 does an AddHandler php5-script .php which tells Apache to run all files with a .php extension through the PHP5 interpreter. I didn't know this, but it even does this for files with names like sample.php.html, where .php isn't at the end of the filename. So even though the phpdoc output files should just render as HTML, they were being interpretted as PHP and were throwing errors.

So I created a directory called /var/www/html/phpdoc and added the following to /etc/httpd/conf.d/php.conf in a Directory container (and restarted Apache): RemoveHandler .php

That convinced Apache not to run any files in that directory through the PHP5 interpreter. Incidentally, I had previously tried SetHandler default-handler for that directory, and it disabled PHP5, but it also disabled nice things like autoindexing (which broke URLs like http://localhost/phpdoc/sample/: Apache would refuse to serve a directory).

By the way, this doesn't seem to affect CentOS4/RHEL4 (Apache 2.0.x and PHP4), because Apache sets up PHP a little differently: it does an AddType, so there's no conflict of having both a text/html content type and a PHP5 handler.

XML in PHP5: the weather

My favorite weather-related Web site is the weather underground, but their pages can be a bit heavy. Usually I just want a quick summary of current conditions and a forecast for the next day or two. Thankfully, wunderground provides this in XML format. Here's the example for Portland, Oregon: HTML, XML.

I thought it would be fun to write a quick PHP program to download the XML file, parse it, and present it in an easy-to-read format. I decided to use the SimpleXML extension for PHP5, because my XML-parsing needs are pretty modest for this project. And I'll use the curl extension to fetch the XML file.

$url = 'http://rss.wunderground.com/auto/rss_full'
  . '/OR/Portland.xml?units=both';

$ch = curl_init($url);
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 );
curl_setopt( $ch, CURLOPT_HEADER, 0 );
$xmlstr = curl_exec($ch);
$res_info = curl_getinfo($ch);
curl_close($ch);
if ( $res_info['http_code'] != 200 ) {
  header( 'content-type: text/plain' );
  die("couldn't open $url");
}

$xml = new SimpleXMLElement($xmlstr);
$epoch = strtotime( $xml->channel->pubDate );
$date = date( 'H:i:s l j F Y', $epoch );
$report_uri = htmlentities(
  $xml->channel->item[0]->link );
$content = '';
$forecast_items = array();
foreach ( $xml->channel->item as $item ) {
$desc = strip_tags( $item->description );
$forecast_items[] = array(
  'guid' => $item->guid,
  'desc' => htmlentities( html_entity_decode($desc) ),
);
}

echo '<html><body><h1>Weather Underground Report</h1>',
  '<h2>Portland, Oregon: ', $date, '</h2>';
foreach ( $forecast_items as $item ) {
  $id = '';
  if ( !empty($item['guid']) ) {
      $id = ' id="' . $item['guid'] . '"';
  }
  echo "<p$id>", $item['desc'], '</p>';
}
echo '<p><a href="', $report_uri, '">Full report</a></p>',
  '</body></html>';

There's some magic in the first foreach loop. Just as you should never trust anything typed into a Web form, you should also be skeptical of content from a foreign XML document, hence the strip_tags() and htmlentities() calls. But some of the characters in the wunderground XML are already HTML-encoded (like the degree symbol), so it's useful to call html_entity_decode() first (otherwise the temperature might look like "75°F", rather than "75°F").

The code is otherwise straightforward. If you look at the raw XML, you'll find that the entire report is wrapped in a <channel> container, inside which the report date is wrapped in a <pubDate> container, etc. As its name implies, SimpleXML makes parsing XML pretty easy, and it's a great choice for small projects like this.

fopen($url) v. curl in PHP

Occasionally you'll see PHP code which uses require() or include() or fopen() or file_get_contents() to import code from a remote location (the argument to those functions can be a URL). This sort of thing is generally considered to be a bad security practice, especially if you don't control the code at the remote location (it could unexpectedly change in such a way as to do something destructive to your application).

Many PHP security experts tend to recommend disabling the allow_url_fopen option in php.ini. Disabling this feature can even serve to prevent inadvertent code injection. Imagine an application which calls require($file) where $file is dynamically determined. If your application has some sort of problem which allows an attacker to set the value of $file, the attacker can inject the code of his/her choosing into your application.

So I feel that disabling allow_url_fopen is a good idea, but sometimes you need to initiate HTTP requests in your PHP code. The curl extension provides a good way of doing this. The following snippet will put the contents of the Web page at $url into the $page variable:

$ch = curl_init($url);
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 );
curl_setopt( $ch, CURLOPT_HEADER, 0 );
$page = curl_exec($ch);
curl_close($ch);

The previous example is a GET request, but you can also do POST:

$postdata = 'var1=value1&var2=value2';
$ch = curl_init($url);
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 );
curl_setopt( $ch, CURLOPT_HEADER, 0 );
curl_setopt( $ch, CURLOPT_POSTFIELDS, $postdata );
$page = curl_exec($ch);
curl_close($ch);

undeclared attributes in PHP classes

I was recently surprised to discover that a PHP object can have attributes (variables) not declared in the class. The following works in PHP4 and PHP5 (it prints 'ick' and then 'yark'):

class Gakkk {}

$gakkk = new Gakkk();
$gakkk->blech = 'ick';
echo $gakkk->blech, "\n";
$gakkk->blech = 'yark';
echo $gakkk->blech, "\n";

At best, this strikes me as a very poor programming practice. Who knows when this sort of thing will stop working (in a future version of PHP)? And what kind of code readability is this?

I really like PHP, but this is just weird to me.

PHP4 in RHEL4

Since last year's announcement that PHP4 will reach end-of-life this summer, I've been wondering what will become of PHP in the versions of Red Hat's Enterprise Linux distribution which shipped with PHP4. Looks like Red Hat will continue to provide bugfix and security updates for PHP4 throughout the lifetime of its PHP4-relevant distributions.

IntranetAddress PHP class

I've added another Google code project. This one is called IntranetAddress, and it's a PHP class which you can use to determine whether or not an IPv4 address belongs to a set of network ranges (specified in CIDR notation in a configuration file). The class requires the Net::IPv4 PEAR package, and a PHPUnit test suite in included.

Inspekt PHP library

A recent post on the Planet-Websecurity.org blog got me interested in Inspekt. It's a secure input validation library for PHP. It reminds me a bit of Perl's taint switch, in that Inspekt prevents you from directly using $_POST, $_GET, and their ilk.

Looks like it hasn't really hit release status yet, but I think it's going to be worth watching.

PHP4 end of life

I maintain a lot of legacy PHP code on some CentOS 4 servers, and CentOS 4 comes with PHP4 (I only recently became aware of the PHP5 packages in the centosplus repository). I've long resisted trying to move to PHP5 due to (probably overblown) fears of broken code.

PHP recently announced the PHP4 end of life at the end of 2007 (with some security updates through 8 August 2008). So it looks like I've just about run out of excuses.

Makes me wonder what Red Hat will do about their RHEL3 and RHEL4 distributions.