Lazarus Cross-platform Fullscreen Support

I’m currently working on fast bitmap techniques for Lazarus, for which I will be posting a separate article in the future. However, whilst working on this I have encountered an old bug-bear of mine, implementing full-screen support. If your not familiar with the concept, most browsers now support this mode of operation, where they can take over use of the whole screen, removing all traces of the operating system from the display. This is also common in computer games and video players. It is very likely that the browser you are reading this in supports this feature. On PCs pressing F11 should swap into and out of fullscreen mode.

In Lazarus’ LCL library there is now an additional WindowState property for forms of wsFullScreen. This aims to simplify the process of taking over control of the whole display in a cross-platform manner for all LCL based applications. A naive use of this technique within a Lazarus application would look like this:

Unit Main;

{$mode objfpc}{$H+}

Interface

Uses
  Classes, SysUtils, FileUtil, Forms,
  Controls, Graphics, Dialogs, StdCtrls;

Type
  TMainForm = Class(TForm)
    FullscreenButton: TButton;
    RestoreButton: TButton;
    Procedure FullscreenButtonClick(Sender: TObject);
    Procedure RestoreButtonClick(Sender: TObject);
  End;

Var
  MainForm: TMainForm;

Implementation

{$R *.lfm}

Procedure TMainForm.FullscreenButtonClick(Sender: TObject);
Begin
  WindowState := wsFullScreen;
End;

Procedure TMainForm.RestoreButtonClick(Sender: TObject);
Begin
  WindowState := wsNormal;
End;

End.


This works perfectly on Linux(QT), but on Windows the form border is not hidden in fullscreen mode and worse still Linux(GTK2) completely fails to restore the window. No doubt these are teething problems that will be resolved in due course, but it is frustrating if you want to create a reliable cross-platform solution using the LCL framework. Fortunately, there are a couple of simple tweaks that can be applied to this code to make it work reliably across the main Lazarus target platforms.

Firstly, the GTK2 failure to restore seems to be due to the WindowState change not actually being applied. The trick here is to change the state to maximized immediately before the change to normal. This persuades GTK2 to do the right thing without adversely effecting the other platforms. So the restore procedure now looks like this:


Procedure TMainForm.RestoreButtonClick(Sender: TObject);
Begin
  WindowState := wsMaximized;
  WindowState := wsNormal;
End;

To deal with the failure to remove the border on Windows, we have to do that explicitly using the BorderStyle property. This leads to the procedures looking like this:


Procedure TMainForm.FullscreenButtonClick(Sender: TObject);
Begin
  BorderStyle := bsNone;
  WindowState := wsFullScreen;
End;

Procedure TMainForm.RestoreButtonClick(Sender: TObject);
Begin
  WindowState := wsMaximized;
  WindowState := wsNormal;
  BorderStyle := bsSizeable;
End;

Unfortunately, changing the border style interferes with the position of the restored window. So to tidy this up, the BoundsRect of the window needs to be recorded prior to changing to fullscreen, then set again upon restoration. I like to wrap this behaviour in an auto-checking Action so that it can be associated with a keyboard short cut (F11). Thus, the final routine I use in production code looks like this:


Var
  WindowRect: TRect;

Procedure TMainForm.FullscreenActionExecute(Sender: TObject);
Begin
  Begin
    If FullscreenAction.Checked Then
      Begin
        WindowRect := BoundsRect;
        BorderStyle := bsNone;
        WindowState := wsFullScreen;
      End
    Else
      Begin
        BorderStyle := bsSizeable;
        WindowState := wsMaximized;
        WindowState := wsNormal;
        BoundsRect := WindowRect;
      End;
  End;
End; 

Counting Intel CPU Clock Ticks

Accurately timing sections of a program for performance profiling is a surprisingly error prone task.  Whilst timer circuitry exists on all modern PC motherboards, it is designed for long duration use and can only perform accurate timing in the millisecond range or longer.  Modern CPUs execute at gigahertz frequencies and therefore need timing capabilities in the nanosecond range to be useful for code profiling.

The obvious solution is to read the CPU’s own internal clock tick counter as this is the actual mechanism used by the chip itself.  Fortunately, Intel have exposed this tick count through the rdtsc instruction, which returns the CPU tick count since the chip was powered up.  It is tempting to imagine that dividing this number by the processor’s clock frequency will yield a nanosecond precise time value.  This used to be true for older generations of Intel CPU’s running single process operating systems, but this is no longer the case.

Firstly, there is a conflict of interest between power use and performance.  Chips consume energy for every clock cycle completed, so chip makers have developed sophisticated methods of adjusting the actual clock speed depending upon current system load.  As a result, it is not possible to be certain of the clock frequency in effect at any given moment, even if the exact model of the CPU is known to the program.

Secondly, on modern pre-emptive multitasking operating systems, any given section of code may be paused at any point for small periods by the operating system as it allocates CPU resources to other tasks.  This means that not all of the clock ticks recorded will necessarily have been used by the code being observed.

These limitations have to be understood and the solution from a profiling point of view comes firstly from the observation that the the actual real time value isn’t important for comparing code. The number of clock cycles is, so changes to code that reduce the number of clock cycles required to perform a task are always going to yield a performance improvement, even if that cannot be precisely timed. The impact of the second problem can be reduced by attempting to quantify level of uncertainty in the observations recorded. This situation can be expressed mathematically in the following simple equation:

ObservedTicks = ActualTicks+Error

The actual clock cycles required for a given sequence of instructions are invariant. Therefore if multiple observations are made it follows that:

ObservedTicksmin = ActualTicks+Errormin

This implies that the error can be minimised by increasing the number of observations until there is no further reduction in the minimum observed count.  At this point, the error is either zero or some residual constant which can’t be altered from within our code anyway. This means that whilst a single measurement cannot be relied upon for profiling, an appropriately sized set of observations can give a useful benchmark value.

To be able to profile my own Free Pascal code, I have developed a small unit to make use of the rdtsc instruction. It contains 32 and 64 bit Intel assembler so that it can be used on both the 32 and 64 bit versions of the compiler with no operating system dependencies (It has been tested on Windows, Linux and Mac OS X). An example of its use is shown in the following code snippet:


Procedure DrawImage;
Begin
  CPUTimer.Start;
  Canvas.Draw(0, 0, ImageBuffer);  { The code being profiled. }
  CPUTimer.Stop;
  WriteLn('Image Draw Time: '+CPUTimer.MegaCountAsText+'MTicks');
End;

The timer is created as a singleton global object, so it does not need to be created or freed. There are three helper methods that return the tick count as text. Given that these values can often be very large numbers, two of these output routines perform divisions on the count by 1,000 and 1,000,000 respectively to make display more manageable.

Given that the code is quite short I have pasted it here in full:

Unit CPUTime;

{$IFDEF FPC}
  {$ASMMODE INTEL}
  {$MODE OBJFPC}
  {$LONGSTRINGS ON}
{$ENDIF}

Interface

Uses
  Classes, SysUtils;

Type TTickUnits = (tuTicks, tuKiloTicks, tuMegaTicks);

Type
  TCPUTimer = Packed Object
    StartTick: Int64;
    TickCount: Int64;
    Function FormatTickCount(Count: Int64; Units: TTickUnits = tuTicks; ShowUnits: Boolean = False): String;
    Function KiloCountAsText: String;
    Function MegaCountAsText: String;
    Function TickCountAsText: String;
    Procedure Mark; Assembler;
    Procedure Start; Assembler;
    Procedure Stop; Assembler;
  End;

Var
  CPUTimer: TCPUTimer;

Implementation

Function TCPUTimer.FormatTickCount(Count: Int64; Units: TTickUnits = tuTicks; ShowUnits: Boolean = False): String;
Var
  DisplayTicks: Int64;
  UnitSuffix: String;
Begin
  UnitSuffix := EmptyStr;
  Case Units Of
  tuTicks:
    Begin
      DisplayTicks := Count;
      If ShowUnits Then
        UnitSuffix := 't';
    End;
  tuKiloTicks:
    Begin
      DisplayTicks := Count div 1000;
      If ShowUnits Then
        UnitSuffix := 'Kt';
    End;
  tuMegaTicks:
    Begin
      DisplayTicks := Count div 1000000;
      If ShowUnits Then
        UnitSuffix := 'Mt';
    End;
  End;
  Result := IntToStr(DisplayTicks div 1000)+UnitSuffix;
End;

Function TCPUTimer.KiloCountAsText: String;
Begin
  Result := FormatTickCount(TickCount, tuKiloTicks, True);
End;

Function TCPUTimer.MegaCountAsText: String;
Begin
  Result := FormatTickCount(TickCount, tuMegaTicks, True);
End;

Function TCPUTimer.TickCountAsText: String;
Begin
  Result := IntToStr(TickCount);
End;

Procedure TCPUTimer.Mark; Assembler; NoStackFrame;
Asm
  {$IFDEF CPU64}
  mov rcx, Self
  rdtsc
  shl rdx, 32
  or rax, rdx
  sub rax, rcx[StartTick]  
  mov rcx[TickCount], rax
  {$ELSE CPU32}
  mov ecx, Self
  rdtsc
  sub eax, ecx[StartTick]  { Subtract low 32bits first. }
  sbb edx, ecx[StartTick+4]  { Subtract high 32bits with borrow. }
  mov ecx[TickCount], eax
  mov ecx[TickCount+4], edx
  {$ENDIF CPU64}
End;

Procedure TCPUTimer.Start; Assembler; NoStackFrame;
Asm
  {$IFDEF CPU64}
  mov rcx, Self
  rdtsc
  shl rdx, 32
  or rax, rdx
  mov rcx[StartTick], rax
  {$ELSE CPU32}
  mov ecx, Self
  rdtsc
  mov ecx[StartTick], eax
  mov ecx[StartTick+4], edx
  {$ENDIF CPU64}
End;

Procedure TCPUTimer.Stop; Assembler; NoStackFrame;
Asm
  {$IFDEF CPU64}
  mov rcx, Self
  rdtsc
  shl rdx, 32
  or rax, rdx
  sub rax, rcx[StartTick]
  mov rcx[TickCount], rax
  mov rcx[StartTick], 0
  {$ELSE CPU32}
  mov ecx, Self
  rdtsc
  sub eax, ecx[StartTick]  { Subtract low 32bits first. }
  sbb edx, ecx[StartTick+4]  { Subtract high 32bits with borrow. }
  mov ecx[TickCount], eax
  mov ecx[TickCount+4], edx
  mov ecx[StartTick], 0
  mov ecx[StartTick+4], 0
  {$ENDIF CPU64}
End;

End.


FreePascal 2.7.1 on Raspberry Pi

[EDIT: Please look here for more up-to-date instructions for Free Pascal 3.1.1 with Lazarus 1.7]

Since I wrote my original article about building FreePascal with Lazarus on the Raspberry Pi there have been a lot of developments.  One of the challenges with this platform is that the rapid pace of change very quickly makes a published set of instructions obsolete. This is now true of the previous scripts I have released which will not build the current trunk source code.

In my last post I covered how I now connect to the Pi from my laptop, which I find a lot more convenient.  In this post I will explain the current method I am using to build an up-to-date FreePascal compiler from the source code available from the official subversion repository.  Whilst the original compiler I built still works on the current release of Raspian, it does not contain many of the recent advances made by the FreePascal developers over the last year, or so.  The good news is that they have added a lot of new features that improve the quality of code generation for ARM chips.  This work has been the result of a concerted effort to support Android devices, but it also benefits developers wanting to target Raspberry Pis, making it a very attractive development language for the Pi.

The standard method for installing FreePascal from the Raspian repositories still works as Debian administrators would expect:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install fpc

However, this installs FreePascal 2.6.0. and since the current stable release is 2.6.4 this is now looking very dated.  The current development release, the one with the useful new features, is 2.7.1.

The source code for 2.7.1 can be downloaded using subversion, which needs to be installed before attempting to follow these instructions:

 sudo apt-get install subversion

The source code can then be retrieved using the following command.  I create a folder called FreePascal that I used to contain the various sources I use and the following command should be executed from there:

svn co http://svn.freepascal.org/svn/fpc/trunk 2.7.1

Also, if you are using a new install of Raspian, the default 2GB partition size is far to small for building FreePascal, I would recommend resizing the partition to take advantage of any additional space available on your SD card using the ‘raspi-config’ utility.

To be able to build 2.7.1, the required seed compiler is currently 2.6.4.  So to do this properly, we would need to build several sucessive releases of FreePascal to match these build requirements, which would be a very time consuming process.  Fortunately, I am not the only person interested in building FreePascal on the Pi; a very useful recent contribution was made by Thaddy de Koning.  He has built a 2.7.1 seed compiler which he has published here.  It is currently only a month old and he has also detailed the build method he uses here, the actual make command he uses is:

sudo make clean all distinstall OPT="-dREVINC -dFPC_ARMHF -CX 
-CpARMV6 -CfVFPV2 -CaEABIHF -OpARMV6 -O2 -OoFASTMATH -XX -Xs"

This assumes there is the required 2.6.4 compiler installed.  This is clearly not the case if you have a bare Raspian image installed.  Fortunately you can pass an option to the make script indicating the location of an unistalled seed compiler.  I extracted Thaddy’s actual compiler binary ‘lib/fpc/2.7.1/ppcarm’ from the above archive and copied it to my FreePascal working folder on the Pi.  I also renamed it ‘thaddyppcarm’ to avoid confusion once I start creating my own copies of this file.

Unfortunately, whilst trying to execute this process, I ran into a couple of problems. Firstly, as mentioned, the current make scripts expect 2.6.4 as the seed compiler.  Since Thaddy’s is later than this there is no problem with using it, except that the make script now performs version checking.  This can be suppressed by adding ‘OVERRIDEVERSIONCHECK=1’ to the command line.  The second problem was that the build process would halt with the following error:

Fatal: Can not open include file "revision.inc"

I don’t know why this should fail, or what the expected behaviour should be, but I found that process would complete successfully if the ‘REVINC’ option was removed from the command line.  So the actual make command that worked for me was:

sudo make clean all distinstall OPT="-dFPC_ARMHF -CX -CpARMV6 
-CfVFPV2 -CaEABIHF -OpARMV6 -O2 -OoFASTMATH -XX -Xs" 
OVERRIDEVERSIONCHECK=1 FPC=/home/pi/FreePascal/thaddyppcarm

Incidentally, the other options on this command line enable the additional processor support I mentioned earlier.  Most notably ‘FPC_ARMHF’ enables hardware floating point which will make a huge difference to the performance of computationally intensive programs.

To install the new compiler to its usual location I then use the same sequence of commands as given in the original scripts.  So copying the compiler and recreating it’s global symlinks requires:

sudo make install PP=compiler/ppcarm PREFIX=/usr
sudo rm -f /usr/bin/ppcarm
sudo ln -s /usr/lib/fpc/2.7.1/ppcarm /usr/bin/ppcarm

After which you will need to do the same for the compiler source:

sudo make install sourceinstall PREFIX=/usr
sudo rm -f /usr/share/fpcsrc
sudo ln -sf /usr/share/src/fpc-2.7.1/fpc /usr/share/fpcsrc

Lastly the fpc configuration file needs to be created with:

sudo /usr/lib/fpc/2.7.1/samplecfg /usr/lib/fpc/2.7.1 /etc

After completing all of this you should have a fully installed FreePascal Arm 6 compiler with hard float support.  The screen shot below shows this compiler being used to build a test X11 program which ran successfully on the Raspberry Pi over an SSH connection from my laptop:

FreePascalCommandLine

I have also updated the original build script to automate this process of rebuilding the compiler from the main subversion repository and it is available for download here.  At the moment, I do not need Lazarus on the Raspberry Pi, but it should be possible build it following my original instructions using this compiler instead.  One further thing to note; if you have not installed the fpc 2.6.0 from the repository, you will not have some of the required developer packages installed and will therefore have to install these manually.  For example the X11 test program cannot be linked without installing the ‘xorg-dev’ package with:

sudo apt-get install xorg-dev

 

Return to the Raspberry Pi!

Since I published my last post on the fencing score box system I have been contacted by a hardware manufacturer who is interested in embedding a Raspberry Pi in their new range of equipment.  As a result, this has put the Pi firmly back on my agenda!  I will return to the build process for the FreePascal compiler in another article, but before then I want to cover the method I am now using to connect to the Pi as I think it is far more convenient for a developer, than having the Pi set-up with a dedicated screen, keyboard and mouse.

The basic idea is to create a local subnet via a direct cable connection to the Pi and then interact with the Pi via SSH using the laptops own peripherals.  There are quite a number of ways of doing this, but I will describe the process I use as I think it is the simplest to set-up (at least from an Kubuntu/Ubuntu point of view).  To start with the Pi needs to be attached to a power source.  This can be a micro-USB mains charger, or a USB to micro-USB cable to draw the power off the laptop.  If you choose the second method, be sure your laptop is capable of reliably supplying the additional current required.  I prefer to power the Pi independently, to avoid any issue with straining my already hot laptop PSU.

Next the Pi is connect to the laptop via a standard Ethernet cable.  A cross-over cable is not required as the Pi is smart enough to detect this type of connection and reconfigure itself accordingly.  Once plugged in, you will need to know the IP address allocated to the Pi to be able to communicate.  This is where the confusion begins.  There seem to be four approaches to doing this.  Two require editing configuration files on the PI itself.  As with any editing of configuration files, it is always a wise precaution to make a backup copy of the file before making changes!  To edit configuration files on the Pi, I plugged the Raspian SD card directly into my laptop and edited it there before re-inserting it into the Pi.

The first way to allocate an IP address is to edit the ‘/etc/network/interfaces’ file on the Pi to define a static address for the eth0 interface.  This is the first method I tried, but I could not get this to work.  I suspect there are other start-up processes on the Pi that override this Linux configuration method.

After a bit more googling I found the second method.  It is also possible to pass a boot command line argument to the Pi that allocates an IP address.  This is achieved by editing the ‘cmdline.txt’ file in the boot partition of the Pi by adding ‘ip=169.254.0.1’, for example, to the end of this file.  Care must be taken not to insert a return as this must be all on one line.  This enabled a basic SSH connection to the Pi with the command ‘ssh pi@169.254.0.1’. However, the Pi was isolated from the wider network by the host laptop, so for example: ‘sudo apt-get update’ would not connect to the repositories.

The last two solutions solve this problem.  One is to set-up the laptop as a full-blown DHCP server.  This is certainly possible, but I feel it is overkill for what I wanted to achieve.  The last solution is to use internet connection sharing on the laptop, to allow network traffic to be passed through the laptop’s network connections.  My laptop uses an internal WIFI card to connect to our home ADSL rooter, or at work, the company’s wireless network.  This leaves the Ethernet port free to connect to the Pi. Internet connection sharing, permits traffic to be passed from one interface to the other.  In Kubuntu, and I believe Ubuntu also, this can quickly be set through the network connection editor.

ConnectionPropertiesThe default mode for cable connections is ‘Automatic’ on the ‘IPv4’ tab.  Changing this to ‘Shared to other computers’ sets up the interface for internet connection sharing.  I also set the option to ignore ‘IPv6’ connections as I did find an article which suggests this can cause problems with a Pi, although I don’t know if that is really required.  I also switched off automatic connection as I want to be able to manually choose between this and a standard DCHP connection.

ConnectionEditor

I also discovered that once internet sharing was enabled, it was not necessary to set the boot parameter IP address as one was allocated anyway.  In my case it was ‘10.42.0.44’. This can be discovered with the arp command.  Knowing this, I then edited the ‘/etc/hosts’ file on my laptop to look like this:

127.0.0.1 localhost
127.0.1.1 laptop01
10.42.0.44 rpi

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

This enabled the SSH command to connect using the name ‘rpi’ rather than having to remember the IP address.  Its worth pointing out two useful features of SSH for those that are new to it.  The first is the ‘-X’ parameter.  This enables remote X Windows applications to be started over SSH.  The second is that modern Linux desktops can use it as a networking protocol to provide remote folders in their visual file managers.

Desktop

In this screenshot I have started the Pi’s Midori browser via an X Windows link and opened a Dolphin file browser looking at the Pi’s FreePascal folder.  So for Linux the full SSH command line for the default pi user looks like this:

ssh -X pi@rpi

Its worth mentioning for Windows users, that they can also use SSH by installing a utility called Putty.  I have not used this myself, but I have read several articles where this has been successfully used to achieve the same working environment as this.  At some point, I will also investigate using RSA keys which may be combined with SSH to avoid having to use passwords every time you connect.

Software Fencing Score Box

I have just started a new open source project to create a fencing score box system.  The idea is to use an old laptop or other low cost system to emulate the hardware of an electric fencing score box.  My son has been competitively fencing for just over a year now and I have been looking for an affordable way of enabling him to train at home.  The best value for money system I have come across is the one used by his coach, the Epee Hitmate.

However, that is still somewhat out of our price range at present.  As a result, I have started investigating the possibility of emulating the control box hardware with software and then interfacing this to the actual weapon switches.  The ideal solution would be to connect wirelessly to something like the HitMate’s transmitter units, but failing that a conventional wire spool connection would do.

I have also noticed that fencing referees are increasingly using smart-phone apps to run bouts and I thought it would be an interesting exercise to see if a smart-phone could be used as a remote control for such a project.  To avoid having to develop a version for each OS, I decided to embed a small webserver so that any mobile browser app could be used as the controller.  The screen shot below shows this running on an iPhone simulator:

iPhoneBrowser

For those unfamiliar with fencing, the basic information is quite simple:  there is a score for each fencer, one to the left of the referee (red) and one to the right (green) and there is also a count down of the time remaining in the current bout.  The buttons enable the referee to start and stop the timer and adjust the scores for left and right.

The main program is a conventional application written in FreePascal.  This can be compiled to run on Linux or Windows (and possibly a Mac as well!) and has no external dependencies, although you can add an optional WAV file if you want a buzzer to sound during a hit. The following screenshot shows this running on Linux :

RaspberryPi

In fact, the keen eyed amongst you will probably have noticed the little perspex box and realised this is actually running on a Raspberry PI!  This was compiled with the PI FreePascal compiler I created last year.  To be honest, I haven’t done anything with the RPI since then as, until now, I had not thought of a practical use for it.  However, the PI is actually an ideal platform for this project.  Also the world of FreePascal on the PI has moved on tremendously and there really isn’t any need to build the compiler from source now.  Further more, Thaddy de Koning has created a Pascal interface for the PI’s OpenVG library, giving access to the powerful GPU for 2D graphics.  This will go a long way to compensate for the PI’s somewhat underpowered CPU.

So if you would like to take a look at this project, it is available on Bitbucket:

https://bitbucket.org/PaulFMichell/allez/

A pre-compiled windows binary is available here:

https://bitbucket.org/PaulFMichell/allez/downloads/Allez.exe

If you want to test the embedded webserver it  connects on port 8080 to aviod clashing with any other webserver that might be running on the default port 80.  So to access it on the local machine requires the following URL:

http://localhost:8080/

This is still very much a test system and it is a long way from being a completed project. However, it does demonstrate some of the capabilities of FreePascal and will serve as a good starting point for a more focused endeavour and if there are any Fencing – Electronics – Raspberry PI enthusiasts out there who have any bright ideas about linking the weapon switches, I would love to hear from you!

 

 

OS Meridian Roads Data

I have been doing a lot of work processing OS OpenData recently and whilst loading Meridian 2 onto a PostGIS system, I have encountered a couple of issues that I thought were worthy of a post.

 

The source data files I am working with are the latest MapInfo TAB files (Version: 07/2013) downloaded from the OpenData website.  My working system is a Kubuntu 13.04 laptop and the Server a virtual machine running PostgreSQL 9.1.  The actual data transfer is performed by the Geospatial Data Abstraction Library (GDAL).  I have written a short BASH script that calls the GDAL ogr2ogr utility, iterating over the extracted files from the downloaded archive:

#!/bin/bash 
function UploadFiles
{
 # Upload TAB files to PostGIS with OGR.
 for File in "$1"/*.TAB 
   do
     if test -f "$File"
     then
       echo "$File"'...'
       export PGCLIENTENCODING=LATIN1
       ogr2ogr -f "PostgreSQL" PG:"host=XXX.XXX.XXX.XXX 
       user=postgres dbname=OpenData password=XXXXX port=5432"
       "$File" -a_srs "EPSG:27700" -skipfailures -progress
     fi
   done
 # recurse directories
 for SubDir in "$1"/*
   do
     if test -d "$SubDir"
     then
       UploadFiles "$SubDir"
     fi
   done
}
UploadFiles "/media/Archive/GIS Data/OS OpenData/Meridian"
However, I did encounter a few snags.  Firstly, the A road section ‘O13RDLWRD2HTV’ caused the following warning:
Warning 1: Geometry to be inserted is of type Multi Line String, whereas the layer geometry type is Line String.

Converting the geometry for this record into well known text format revealed:MULTILINESTRING((430708.0001 431916.9998,430793.00005 431966.00005),(430791.00015 431966.00005,430837.9999 431994.0001))The Meridian road network is intended to consist of continuous lines, so this two part record is, in effect, a broken line.  The work around is to remove the inner line ends and convert the remaining end points into a single line string:LINESTRING(430708.0001 431916.9998, 430837.9999 431994.0001)To help visualise this I have prepared a graphic using Quantum GIS. The original broken line is rendered in red and my suggested solution in green.

MeridianBug

The fixed record can be added back onto the server with the following SQL command:

INSERT INTO "a_road" 
("wkb_geometry" , "feature_code", "osodr", "route_number", 
 "road_name", "length_metres", "trunk_indicator") 
VALUES 
(ST_GeomFromEWKT('SRID=27700;LINESTRING(430708.0001 431916.9998, 
 430837.9999 431994.0001)'), 3001, 'O13RDLWRD2HTV', 'A61', '', 
 152, '');

There is a second similar problem in the minor roads layer for road section ‘O16A01KU1ABTW’.  The solution for this is exactly the same, stitching the two segments back into a single line.  I have reported these two issues to Ordnance Survey and  I have received feedback from the Product Manager that both errors have now been fixed. The changes will be issued in the next release of Meridian 2 Open Data.

One last thing to mention, on my first attempt at the data upload, I did find some accented characters had not been correctly loaded.  The PGCLIENTENCODING define in the BASH script hints PostgreSQL to perform the correct character re-encoding during the import.

Lazarus on the Raspberry Pi

[EDIT: The build steps detailed in this post will no-longer work with current (10/07/14) versions of Raspbian and Free Pascal. Please see the more recent post here for a newer set of build instructions for Free Pascal 2.7.1 and here for Free Pascal 3.1.1 with Lazarus 1.7]

I have been investigating the use of Free Pascal and Lazarus on the Raspberry Pi. The Pi is a minimal, low cost, ARM based computer, designed primarily for educational uses. As a Pascal advocate, I would love to see a modern Pascal compiler made widely available on a platform like this.

The official operating system for the Pi is called Raspbian which, as it’s name suggests, is a derivative of Debian Linux. Free Pascal and Lazarus packages are available on Debian, so I wanted to see if they had successfully transferred to Raspbian in a usable form.

I have one of the new Raspberry Pi Model B’s that has 512Mb of RAM and I have been using it with the current Raspbian image (2012-10-28). After spending a little while getting to grips with Raspbian, I was able to progress to the business of testing Free Pascal. The good news is that the standard Debian packaging commands worked a treat:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install fpc
sudo apt-get install lazarus

The only snag from a visibility point of view is that there is no automatic integration of Lazarus into the LXDE menu. From the command-line though, ‘fpc’ will invoke the compiler and ‘lazarus’ the GTK2 based IDE. However, as with Debian, the versions of the available packages are a little behind the current releases. On my main Kubuntu system, I habitually build Free Pascal and Lazarus from source, using Subversion to obtain the latest changes. This process requires a seed compiler, which in this case, can easily be supplied from the default repository.

So I started again with a fresh Raspbian SD card, but this time, I only installed FPC from the repository. Free Pascal additionally requires the GNU build tools and fortunately these are already installed in Raspbian, so I only had to add Subversion before I could attempt to use my usual build process on the Pi with a few of the options tweaked for ARM output:

sudo apt-get install subversion

Whilst researching this I have found that others have already attempted this with varying degrees of success. Common problems encountered seem to be caused by running out of memory, or card storage space during the build process. Indeed, I hit both issues even with the Model B’s 512Mb RAM. In the case of RAM usage, I had to reduce the amount of RAM dedicated to the GPU from 128Mb to 64MB. I have seen reports for Model A Pi’s where this needed to be reduced to 16Mb to avoid problems. I also ran out of space on a 4Gb SD card and had to swap to an 8Gb one to provide enough head-room.

After a couple of hours solid processing I had a working build of Free Pascal 2.7.1 with ARM hard float support and Lazarus 1.1. For thoses interested in repeating this process you can download the scripts here: FPC Build Script and Lazarus Build Script. It is worth pointing out that the two scripts behave in slightly different ways. The Free Pascal process is more elaborate for two reasons. Firstly, it always makes a temporary working copy to perform the build, this is a brute force approach to preventing changes over time causing build issues for the compiler or runtime libraries (admittedly at the cost of build time). Secondly, it also performs a full install to the system to superceed the Raspbian 2.6.0 seed compiler. This means that it becomes the new default ‘fpc’ system command. The Lazarus script, by contrast, is a much simpler affair, building its binary in the subversion working folder. I usually run it from this personal folder, from which I can make changes such as installing additional packages, without having to use sudo permissions.

Now I wanted to evaluate the capability of this new setup. Firstly I built several of the standard example programs supplied with Lazarus. All of the ones I tested executed without problems and they looked pretty good on the LXDE desktop. It is worth commenting on the fact that the general operation feels sluggish compared to an Intel based system. However, it is not that slow that you would find it unusable and I have seen Eclipse run slower on a much more capable PC!

Much of my work with Lazarus involves creating graphical output, so I wanted to see how it coped with using some of the graphics libraries available. The built in FPGraphics and LCL TBitmap work without issue, but are often much slower than more specialised options. Next I tried the Lazarus OpenGL package. This unfortunately, did not compile. I will investigate this further and it will probably become the subject of my next post on the Raspberry Pi as the Pi has very good hardware support for OpenGL. I also tried Graphics32 and BGRABitmap which are both highly optimised software renderers for 32bit raster images. Unfortunately, both of them seem to have too many low-level Intel dependencies to port easily to an ARM based system.

To test the system’s graphics capability, I have written a small Mandelbrot set renderer, as shown below, (Source Code). This uses the simplest form of TCanvas.Pixels method for writing the image. This project takes around 2 seconds to generate the image shown below on my Intel i7 based system, but nearly 2 minutes on the Pi! I haven’t performed any detailed bench marking yet, but I will investigate this further.

There is a very interesting Mandelbrot set program called ‘triangle2.c’ supplied on the Raspbian SD image in the /opt/vc/src/hello_pi/ folder. This is written in C and uses OpenGL and fragment shaders to achieve very fast access to the Broadcom GPU hardware present on the Raspberry Pi. It produces stunning results, calculating a full screen Mandelbrot set with overlaid Julia set in real-time. So there should be plenty of scope to reduce my program’s 2 minute render times! It will be especially interesting to see how fast a pure ARM CPU program can operate before having to resort to direct GPU calls for further improvements.

One last comment about Free Pascal is to mention that it can be compiled as a cross compiler, meaning it can be an Intel based program that I can run on my desktop system, but generate ARM binary output for execution on the Pi. Whilst running a native Lazarus desktop on the Pi is perfectly feasible (in fact, it would be a very good platform for introducing people to visual programming in Pascal), large projects would certainly feel more comfortable when worked on with a much faster CPU, especially whilst compiling larger projects and I will definitely be looking into this over the coming weeks.

Lazarus 1.0 Released!

After a 14 year development process the Lazarus project has finally released version 1.0 of this excellent Object Pascal IDE. The length of this extraordinary release cycle is not due to slow progress, but rather a reflection of the very demanding quality standards the core team have held to, refusing to release until they have fixed all bugs classed as required for system stability (many commercial projects could learn a thing or two from this!). Even before this ‘official’ release I have preferred this as my development platform of choice for many years now. If you have used Delphi, you will immediately be at home with this development system. More details are available from the official announcement.

WoodlandCalc Updated

The Woodland Potential Calculator project has just been updated to include the first draft copy of the national dataset:

http://sourceforge.net/projects/woodlandcalc/

There are still a number of areas that have not been individually assessed and some of the NCA descriptions are still only draft place holders. None-the-less, this is a first chance to see the application running with a realistic set of national data. Further updates will follow as the project being lead by the Forestry Commission and Natural England to assess the scope of the potential for increasing the extent of tree cover progresses.

VPR added to Graphics32

Two of my favourite graphics libraries have been merged together. The Vector Polygon Rasteriser (VPR), a very fast polygon scanline converter, has been merged into Graphics32, the software rendering library I used for the display rendering in the Woodland Calculator project.

The old polygon rendering code has been completely replaced and the results are very interesting. The rendering performance is slightly reduced compared with the original fixed point base code, but it is faster than the old floating point, high quality code; whilst managing to produce equally impressive results. In fact, it can perform some tasks, such as rendering thick outlines, much more easily than in the prior version.


Old Renderer

New Renderer

If you would like to see the difference for yourself, you can download the new WoodlandCalc.exe from SourceForge and compare it with the previous one also on SourceForge.