Compiling PHP

Since I've been playing around with computers for more years than I care to remember, I used to be very familiar with the process of unpacking tarballs and compiling code from source. But to be honest it's not something I do very often these days: Most Linux distributions come with an extensive array of pre-compiled binaries: the package management software keeps me up to date with security patches: my C skills are a bit rusty: And life's just too short!

But recently I've been looking at LAMP performance in some detail. I was surprised to find the PHP on my workhorse desktop (PCLinuxOS 2012) had been compiled with no optimization and the resulting binary stripped. I should note that at the time I installed it, there was neither a 64-bit nor an AMD specific port of the installation, hence the OS build was more about compatibility than performance.

So I had a play around to see if there was any benefits at compile time.

PHP is a scripting language and a lot of it's functionality is implemented in extension libraries. Dynamically linking these at runtime does have a performance overhead (although with fastCGI and mod_php, since the code forks rather than loads, this shouldn't be too great). For most people the ability to choose which extensions are loaded at runtime (and thus trim the memory footprint) outweighs the small processing overhead of runtime linking. Unfortunately my test methodology didn't allow me to measure the actual impact of static vs dynamic. In the absence of a site to test and complex tools for HTTP load testing - ab would not cut the mustard - I was using the CLI SAPI where there would be a big performance drop which would not happen on a properly configured webserver.

To compare the different optimization levels I compiled PHP 5.3.22 using gcc 4.5.2 with O0, O2 and O3 then timed 'make test'. My test machine was a dual core AMD athlon

property	O0	O2	O3
CFLAGS	CFLAGS="-march=native -pipe" CXXFLAGS="${CFLAGS}"	CFLAGS="-march=native -O2 -pipe" CXXFLAGS="${CFLAGS}"	CFLAGS="-march=native -O3 -pipe" CXXFLAGS="${CFLAGS}"
Average (sys + usr) seconds	214.0	206.7	207.7
Std Dev (sys + usr) seconds	6.2	0.5	1.0
Max RSS (Kb)	569.8	569.8	570.0
Exe size Kb	6707.9	6880.8	7403.3
size of tripped exe	6257.5	6437.5	6973.6

I've not shown the results here, but I saw no measurable difference between the usr + sys times nor the max rss comparing the stripped and un-stripped binaries.
Interestingly the O3 optimization is very slightly, but measurably slower than O2. And O2 is around 5% faster than O0
The gain of 5% seems a little disappointing compared to metrics reported for other programs but I would expect to see to see greater gains for PHP code implementing (for example) encryption and image processing.

How not to manage an IT dept

I think enough water has flowed under the bridge for me to talk about my experiences as IT manager in a small start up company where I worked until 6 years ago. The characters and events described herein are entirely genuine; only the facts have been changed to protect the innocent.

Having previously worked as Computer Services Manager for a UK retailer, the job sounded like everything I could have wished for - head of IT for a small company producing and selling digital media (that's ringtones for the civilians out there). Hands-on IT work, developing software. Hence I  was happy to accept the job at a significant salary cut from my previous job where I was increasingly just managing contracts.

I joined the company near the end of it's first year of trading when it had a turnover of just £80,000 but promising things were afoot. In the following year, it turned over £2M, then £4M, and was on target for £6M when I left 30 months later. But within a year of my departure it had stopped trading and subsequently wet into liquidation. I'd like to think that I had something to do with its success while I was there. But for now I'll start at the beginning.

In terms of infrastructure they were unusual in doing their own hosting. The implementation of the infrastructure was remarkably well thought out. The code which ran their websites and PRS IVR systems however left a lot to be desired. The code had been delivered an East-European individual who operated through a chain of shell companies. He was then sub-contracting the work to the cheapest bidder. Hence the code was all in different styles and not very sophisticated. However, to be fair, since each page (PHP script) had no dependencies on any other bit of code at least any issues were well isolated, and the code was commented in English.

I was expected to take on the management of the infrastructure and some of the development work, with the plan that I would build a small team of in-house developers. Looking after the servers was easy enough - I made relatively few changes to the configuration. But based on my previous experience, I did spend a lot more time defining and documenting processes than my employers were really comfortable with, although anyone with a background in CMMI / ITIL would probably have said that I didn't go nearly far enough. Along side this I was developing new functionality for the systems and trying to recruit more IT staff.

As this was after the internet bubble had burst, people describing themselves as web developers were ten-a-penny - and that's about all I was able to offer them in a salary. But I really needed competent software engineers / analyst programmers - that the sites were mostly developed in PHP was merely an implementations detail. Hence my first big problem was how to recruit and retain people with real programming skills at half the salary which was available elsewhere.

The story will continue in a future post....

Wifi fettling - rt2870sta / rt5370sta confusion

I'm redecorating my dining room - which is also where my computer lives. Part of this involved re-siting my computer. Rather than run a cable across the room for my internet connection I thought I'd use wireless. I had an old 3com card in my parts drawer - but it appears my kids / the dog had been playing with it. I've always been very happy with 3com kit, but since money is rather tight I decided to go with a basic USB adapter. A quick Google and I found a few people saying that this one worked with Linux.

It's tiny!

It comes with a mini cd which includes source code for Linux drivers! Yeah! Reading the docs, it described how to build and install the RT2870STA kernel module. But my PCLinuxOS 2012 installation  comes with the driver and firmware. So I spent an hour or so trying to get it to work to no avail.

Ho hum, let's try the supplied driver. No configure - just make and make install. But it didn't generate a rt28750sta.ko, instead I got a rt5370sta.ko - it seems that the documentation bundled with the code is out of date. Really I should have gotten the hint when I checked the device with lsusb:

Bus 001 Device 005: ID 148f:5370 Ralink Technology, Corp.

After doing a modprobe I was up and running.

I've disabled the ONBOOT setting for my wired ethernet and enabled it for the new device via drakconf and it all works perfectly (even little green bars in the system tray).

10 feet from the router I get a good signal. I don't know how well it would work at a greater distance with such a small antenna (but our phones get a signal from quite a distance away). It's not really a problem for this machine since it's never going to be very far from the access point.


Link Quality=100/100  Signal level:-52 dBm  Noise level:-83 dBm
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:0  Invalid misc:0   Missed beacon:0

(note to self must remember to add mac address to admin ACL and change the forwarding on access point).

I also got a new desk on Amazon. It's compact and (importantly) feels very solid. The only downsides were that it was a bit tricky to assemble and with my PC on the bottom shelf, the fan is rather noisy (shelf resonates a bit).

Paypal? Privacy? Competence? Apparently not.

Despite my (repeated) warnings, the implementation of the new cookie legislation caught our technical gurus with their figurative pants down. Some disclaimers were cobbled together quickly and posted up.

But it seems we're doing a much better job than Paypal.

Don't get me wrong - I know that (done properly) cookies greatly enhance the security of interaction - indeed I'd be very wary of a payment processing system that didn't implement cookies - but then I'm also very wary of payment processors whom don't implement the law.

Bought some stuff today from Paypal - the first time I'd used Paypal for several months - no mention of cookies. It actually dropped no less than 40 cookies on my browser! FFS! No mention of cookies let alone an opt-in. And while the majority of the cookies were session cookies, it's interesting to note that Paypal DOES NOT END the session at the completion of payment.  That's right. after paying for goods via paypal and returning to the orignal site, you are still logged into Paypal!

"We use cookies written with Flash technology to help prevent fraud "

 Are the criminal classes still that incompetent that don't know how to get around evercookie?

To top it all, it seems that Paypal now stuff more data into their cookies than they are willing to consume. after I wrote this post I went back to double check if there was any effort to comply with The Privacy and Electronic Communications (EC Directive) Regulations 2003, only to get failures accessing https://www.paypal.com/ with an error message indicating that I was returning more cookies than it could cope with:

 (the green bar was added by me to redact my data). It seems that someone thought it a good idea to transfer transactional information via cookies.

Just now the news is full of the arrogance and incompetence of the financial institutions - to a certain extent I have reserved judgement. But increasingly it appears that the IT operations are run by the CEOs nephew who once read a book about programming - OK maybe I exaggerate. There are some very competent and brilliant people out there, some of whom I'm acquainted with - it just seems that the less able IT people of my ken seem to be in the well paid jobs. Grrrrrr!

Very Non Cooperative (really)

The powers that be at my work have decided to revisit the issue of BYOD / external access. I'd previously implemented solutions at two previous employers so thought I'd be able to come up with something suitable here.

In both the previous exercises, I'd arrived at the conclusion that the easiest way to implement this was in terms of services - and relying on open protocols such as HTTP, SMTP, telnet etc (before you reach for your guns, the telnet thing was due to a requirement for dg200 terminal emulation- and I couldn't get a client which would run over ssh - so the telnet was encapsulated in SSL using stunnel). However this project is a bit different for various reasons - not least a very real concerns that the users will leave their computers on trains and park benches. While others in the office had previously come up with horribly complicated encryption schemes - this is a nightmare to support. So after a good deal of thought I realized I could solve a whole load of problems at one stroke by using a remote window / desktop protocol. VNC was the obvious choice due to the wide availability of clients. And I'd previously implemented lots of VNC servers on Linux and on MSWindows - it was always a no-brainer. I did have cunning plans for dealing with small screen real estate, keyboard-less devices etc - but best to start with little steps.

So I fired up synaptic on my PCLinuxOS desktop and installed TightVNC server and a client. Running the server from the command line, it works (but obviously no window manager, and the simple standalone VNC auth). So I set it ll up to run through xinetd (similar to this), re-configure kdm and xfs, fire up a client, enter my username and password - "the server has closed the connection". Check firewall - no problem. Check logs - nothing there. Double check my config changes - all OK. Just to make sure I do clean reboot. Still not working. Check the man page - what's happenned to all the X integration stuff? Gone! No xdm support!

Next I tried RealVNC direct from the RealVNC website - got a licence - read the docs....no more inetd support? WTF? The only logical reason I can think of for this is that they want to enforce their licence terms. Still, I could live with that for the POC - who knows, we might even end up paying for licences for the service - in return for support. But every time I connected to localhost, I got "user not recognised or password was blank". RealVNC do say on their website that this can be an issue on some versions of Linux - and the solution is to disable PAM authentication (a bit weird since they say elsewhere that it is not available in the 'free' version). So I updated the configs, restart the server, to no avail. Tried various tweaks and fettling. Checked the firewall. Nothing. Oh, and there's no 'uninstall' functionality - so had to reverse engineer the installation to clean it up.

Have I got dumber with old age or is this another case where a good product has turned into bloatware?

Aaaaarrrrghhhh!

(some updates added as comments)

Back online

Well, after some digging around I've gone with PCLinuxOS. Flightgear and Google Chrome are available on this. It's significantly faster than Fedora, even though I've switched back to a 32 bit OS (from x86_64). I'd previously installed PCLinuxOS on my daughter's laptop and been impressed with the results.

I've gone with larger fonts (must be getting older) than I had befoe - which combined with the bright colours of KDE makes it a bit cutesy - especially with the MacOS type window decorations.

Kmail still uses Akonadi - but unlike on Fedora, it's not using a mysql backend - and its not maxing out one the CPUs. Still has quite a big footprint - but again no apparent way to switch it off.

I fired up Thunderbird and added the ExportImportTools extension. This still does not support Maildirs - but it was simple to export my mailboxes from kmail and import tham in Thunderbird. The latter integrates rather well with KDE, but it is a bit ugly (unfortunately there are very few themes currently available for v11).

BTW for anyone else wanting to export a mailbox file from kmail...although there are some scripts out there on the internet to do this, if you've got a running kmail installation, all you need to do is select the emails, right click and 'save as...' to export a mailbox format file.

So I'll continue looking for an email client which supports multiple identities / SMTP and POP servers, is lighweight and not ugly. I suspect I'm going to have to wait for a nice Thunderbird theme. I suppose I could write one.....no, I've got enough half-baked projects for the time being.

What were they thinking

My home computer is a gateway to my data (and stuff other people have shared). So the number of X applications I use is short - OpenOffice, Firefox, kmail and konsole, occassionally xrdp and vnc. So I don't upgrade it very often. The last time was when Fedora 9 was shiny and new.

But I wanted to install Flightgear for my son. Rather than go through all the hassle of trying to build it myself, I thought I'd just use binary packages - but none available for Fedora 9. Since it has been a while since I upgraded my operating system I thought I'd just bite the bullet and upgrade to the current Fedora release (16).

A bit of googling and I read about pre-upgrade - this seems like an easy way to upgrade. How wrong was I. It downloaded lots of stuff then rebooted into Anaconda - then stopped - "What type ofmedia contains the installation tree?" - my hard disk. Which hard disk? Again not a problem. What's the path? more reboots and googling and I had a path. Path is invalid. So then I cleaned up all the mess preupgrade had left behind on my disk and tried to upgrade using a CD. You can't upgrade from a CD. Eventually I found the installation DVD iso. Burned a copy and rebooted. "You can only upgrade from the previous 2 releases of fedora". Grrrr!

Time for bed.

Next day. My /home is on a seperate partition and fortunately I had plenty of spare disk. It has been runing on reiserfs for the past 10 years or so - so I created a copy on top of ext4. Then installed Fedora 16, copied over the passwd/shadow/group data, added the ext4 /home to the fstab and rebooted. It seemed to work - but OMG, booting up takes a long time now.

The colours in KDE made some of the text unreadable - fixed that. Load was high - so I found and disabled nepomunk (desktop search engine - no I don't need that). Firefox and OpenOffice running OK. Then I started kmail. Oh dear. The load on my machine went through the roof. WTF is mysqld doing running? I didn't know that kmail now insists on using Anakondi for something - I'm not sure what - certainly I'm pretty sure I don't need it. More googling....apparently since v4.4 of kmail you can disable Anakondi. OK, how do I do that? Use the advanced tab in System Settings. What advanced tab? There is none!

It seems I'm not the only one to be very disappointed in KDE's bloatware.

If I wanted my PC to run very, very slow I would have installed Microsoft Windows on it. I've now spent nearly as much time trying to turn this into a useful computer as I did with MSWindows Vista on my daughter's laptop.

Oh, and kmail failed to important my configuration for SMTP (but didn't think to tell me) and decided to use the local sendmail instead (which I cannot be bothered configuring to handle authentication and TLS nevermind the complications of SPF).

So now wondering if I should switch to different window manager / desktop, or even a different distribution.

Going to a different distribution offers some advantages and will be just as painful as changing window manager. The big issue is migrating my kmail email database and having something which is likely to continue supporting flash for some time. The latter probably means something I can run Google Chrome in. Which narrows down the choices a LOT. I've never been a fan of the way Ubuntu manages permissions - the root user is there for a REASON. Gentoo is fast - and it will run Flightgear and Google Chrome...but it seems like very hard work. Mandriva seems to still be a popular choice - and there are lots of binary rpms for flightgear. However I think Centos looks like a safe option.