From Progzoo
Revision as of 17:03, 19 December 2009 by Andr3w (Talk | contribs) (Hardware)

Jump to: navigation, search

Andrew Cumming runs the server. Andrew's Home Page

How progzoo works

Progzoo relies on WikiMedia (which uses php and mysql), Apache, Perl, XSL, UML.

As a Sysop I can edit

Java:Soak Java:Soak2

Front end

ProgZoo uses mediawiki for the questions. The extension means that authors can include normal wiki markup plus some custom tags. The progzoo extension permits the following tags

   $whoMayNotBeNamed = array();
   $wgParser->setHook( "question", "question" );//Typically include default program and an answer
   $wgParser->setHook( "hint", "hint" );        //User must click to see the hidden text
   $wgParser->setHook( "shell", "shell" );      //Allows program 'shells' to be reused
   $wgParser->setHook( "crash", "crash" );      //A pacman game
   $wgParser->setHook( "soak", "soak" );        //A button to execute all programs on the page
                                                //used to test the serverunder heavy loads
  • ProgZoo extension in /home/andrew/public_html/progzoo/w/extensions
    • progzoo.php - the hook, when a question tag is found it runs them through a couple of xsl files.
      • question.xsl - assembles the user controls (the text box and the Run button)
      • blurb.xsl - the question text is extracted and processed by wiki
  • Zoo skin

The Zoo skin is based on Mozilla's Devmo skin. The Zoo skin has language tabs to allow users to flip between Java, C#, Perl,... It hides most of the Wiki controls. Progzoo 'should' work with other skins.

Back end

A number of servers are used to process users programs. Access to the servers is limited by the queues. Currently there are three queues, javaq, crunchie and cardington, these are represented by the structure $mList which is at the beginning of /home/andrew/public_html/progzoo/r.cgi

my $javaq= {q      =>'java',
            servers=>[{url=>'' ,maxProcesses=>4},
                      {url=>'' ,maxProcesses=>4} ] };
my $crunchie = {q      =>'crunchie',
                servers=>[{url=>"" ,maxProcesses=>1}]};
my $cardington = {q=>'cardington',url=>"" ,maxProcesses=>4};
my $mList = { java=>$javaq, cs=>$crunchie, vb=>$crunchie,

Note that java is the heaviest load, two machines are attached to the javaq. The queuing algorithm will choose the machine with the fewest jobs running (it only counts processes from the queue). The first listed is prefered. Crunchie is dedicated to Windows and so handles C# and VB. The queue Cardington handles all other languages, it shares the second Java machine.

The number of processes maxProcesses allowed is 4 (number of cpu's times 2). The amount of memory dedicated to the UML is 228M. These values took some time to establish, too much memory (512M is too much) and the host falls over, too little and the virtual machine pack up when running java graphics programs (128M is too little).

The UMLs are run by user andrew from a screen session. Use of cow (copy on write) files means that the image pz.fs cannot be damaged.

rm pz.fs.cow;./kernel64-2.6.24 ubda=pz.fs.cow,pz.fs mem=228M eth0=tuntap,tap1,,

UML image The UML image pz.fs includes mysql, java and all the compilers required. It runs apache. The perl script f.cgi compiles and executes the code. Before compiling and executing it uses ulimit to restrict

  • the memory
  • cpu time
  • elapsed time
  • number of processes

These are all very generous but hopefully not fatally so.


The queue is stored in /tmp/ for example /tmp/java and /tmp/crunchie See Panic for details of how to fix broken queues.

Procedure for changing the image

To change the image (if we introduce a new language or library for example) we must start the VM without COW.

#Stop apache in the host...
/etc/init.d/httpd stop
#Switch to the guest screen
screen -R
shutdown -f -h now 
#restart without cow
./kernel64-2.6.24 ubda=pz.fs mem=228M eth0=tuntap,tap1
#Do the update
shutdown -f -h now
#restart WITH cow
rm pz.fs.cow
screen -d -m /home/andrew/kernel64-2.6.24 ubda=/home/andrew/pz.fs.cow,/home/andrew/pz.fs mem=228M eth0=tuntap,tap1 umid=umid
#Detach the screen
Ctrl-A D
#restart apache in the host
/etc/init.d/httpd start

MediaWiki - Changes to LocalSettings.php

$wgExtraNamespaces = array(100=>'Java',110=>'Ruby',120=>'Perl',

$wgNamespacesWithSubpages[NS_MAIN] = true;
foreach ($wgExtraNamespaces as $k=>$v)
  $wgNamespacesWithSubpages[$k] = true;
require_once( "extensions/PathFunctions.php" );




There are whole bunch of servers dedicated (at least partially) to progzoo:

  • sqlzoo5
    • The front end machine, the only one allowed through the Napier firewall, also and and and
  • crunchie (Windows 2003)
  • Fedora 11 - william
  • 146.176.166.??? Fedora 11 - priya (waiting to be allocated an IP address)
  • Fedora 6 - (used for non-java languages - also does the MySQL work for the Wiki)
  • Fedora 11
  • nimbus (Fedora 9)
  • puglia (Fedora 9) - misbehaving cause unknown

What's in the image

Copy the cgi executable (from if 32 bit, is 64 bit)

scp '*' /var/www/cgi-bin/

Make space for copyFiles

mkdir /var/www/cgi-bin/copyFiles
chmod go+w /var/www/cgi-bin/copyFiles

Copy the Java executables (jdk1.6.0_07 is 32 bit, jdk1.6.0_03 is 64 bit)

scp -r /

Must have perl modules:

yum install perl-JSON
yum install perl-DBI

Must turn off selinux

vi /etc/sysconfig/selinux

Set apache to start on boot

chkconfig --level 2345 httpd on

Allow apache through the firewall

vi /etc/sysconfig/iptables

include a line something like

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT


shutdown -r now

Add the new server to the list on /home/andrew/public_html/progzoo/r.cgi

Procedure for adding a new server

On the new server:

#As Andrew - this takes ages so start it first
#Copy the progzoo image from another server
[andrew@puglia ~]$ scp .
#Set apache to autostart, start apache
[root@puglia ~]# chkconfig --level 345 httpd on
[root@puglia ~]# /etc/init.d/httpd start

#Install tunctl and screen
[root@puglia ~]# yum install tunctl
[root@puglia ~]# yum install screen
[root@puglia ~]# yum install perl-JSON
[root@puglia ~]# tunctl -u andrew -t tap1
#Allow port 80 in the firewall - edit /etc/sysconfig/iptables to include a line like
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
[root@puglia ~]# /etc/init.d/iptables restart
#Forward requests to guest to, edit /etc/http/conf/http.conf append...
ProxyPass /guest
[root@puglia ~]# /etc/init.d/httpd restart

iptables -I RH-Firewall-1-INPUT -i tap1 -j ACCEPT
iptables -t nat -A POSTROUTING -s -o eth0 -j MASQUERADE
[root@puglia ~]# chmod og+rw /dev/net/tun
[root@puglia ~]# ifconfig tap1
[root@puglia ~]# echo 1 > /proc/sys/net/ipv4/ip_forward
#As andrew - remove the cow and start in a detatched screen
[andrew@puglia ~]$ rm pz.fs.cow
[andrew@puglia ~]$ screen -d -m /home/andrew/kernel64-2.6.24 ubda=/home/andrew/pz.fs.cow,/home/andrew/pz.fs mem=228M eth0=tuntap,tap1 umid=umid

Make up the file system

Things that can go wrong

No network?

Do ifconfig check that eth0 is there in the guest...

eth0      Link encap:Ethernet  HWaddr 2A:D2:27:57:7E:7E
          inet addr:  Bcast:  Mask:
          RX packets:19 errors:0 dropped:0 overruns:0 frame:0
          TX packets:36 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2045 (1.9 KiB)  TX bytes:4929 (4.8 KiB)

And in that tap1 is in the host

tap1      Link encap:Ethernet  HWaddr 00:FF:21:AF:5B:12
          inet addr:  Bcast:  Mask:
          inet6 addr: fe80::2ff:21ff:feaf:5b12/64 Scope:Link
          RX packets:68 errors:0 dropped:0 overruns:0 frame:0
          TX packets:38 errors:0 dropped:42 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:11816 (11.5 KiB)  TX bytes:5017 (4.8 KiB)

Look at /etc/sysconfig/network-script/ifcfg-eth0 remove from BOOTPROTO dhcp


Turn off udev

To make it boot faster. Switch off in sysconfig, but you have to mount and copy /dev using tar first

The servers are all in B56, Merchiston Campus, Napier University (a Dell, marked server5 or sqlzoo5).