| (no subject) |
[May. 28th, 2009|03:07 am] |
Right, University term has ended, and I want to get some serious progress with lavalamp. Here's hoping. A summary of the night's work:
* Am intended to add support for Zigbee, via the Microchip MRF24J40 2.4Ghz transceiver. This may well require that I rewrite a lot of PIC-side code, but I think it'll be worth it, since the wireless layer I wrote isn't particularly great, and using Zigbee opens the door for some cool interoperability. It does, however, require quite a beefy PIC chip to run the software stack.
* I'm using a PIC18F46K20, instead of an PIC18F4620, mainly because it's cheaper (and more powerful). Hopefully I can make it work. I had to make the following changes to the example code (the co-ordinator project) to change microcontroller:
- in zLink.lkr, the linker script, change
DATABANK NAME=gpr0 START=0x0060 END=0x00FF
to
DATABANK NAME=gpr0 START=0x0080 END=0x00FF
(since bank 1 GPRs start at 0x80 and not 0x60)
and
DATABANK NAME=gpr15 START=0x0F00 END=0x0F7F
ACCESSBANK NAME=accesssfr START=0x0F80 END=0x0FFF PROTECTED
to
DATABANK NAME=gpr15 START=0x0F00 END=0x0F5F
ACCESSBANK NAME=accesssfr START=0x0F60 END=0x0FFF PROTECTED
(since bank 15 GPRs start at 0x0F5F and not 0x0F7F, and SFRs start at 0x0F60.)
also change
FILES p18f4620.lib
to
FILES p18f46K20.lib
- In zNVM.c, change
#if defined(__18F4620) && !defined(USE_EXTERNAL_NVM)
to
#if (defined(__18F4620) || defined(__18F46K20)) && !defined(USE_EXTERNAL_NVM)
- and similarly, in zigbee.def
#if !defined(__18F4620)
to
#if (!defined(__18F4620) && !defined(__18F46K20))
I just know this info will be useful to someone!
* My ICD2 warned me that Vpp was going to go too high for the device. I put a ~8V zener across it, under recommendation from The Internet, and it seems to work.
* I realised that the PIC18F46K20 is a 3.3V device, sometime after applying 5V to it. Ooops.
* I realised that the three-pin 3V3 regulators I have aren't 7805-pinout'ed, after neglecting to read their part numbers and hooking them up as if they were. Burnt my finger.
* Realised that having a pluggable 'transport' layer in lavalamp would be a really damn good idea, as then I can make a 'simple cabled' transport (probs SPI/ttl-level rs232 or the like) for simple stuff, a 'lavalamp native' transport (the one I have been working on to run over any RF Tx/Rx, which currently needs a lot of work) and 'Zigbee' for running Zigbee via the Microchip transcievers. This'd be a lot better than my current design, which has little abstraction between the transport layer and those above. |
|
|
| Videos of recent Uni work |
[Jan. 15th, 2009|07:37 am] |
Since it's the end of Semester 1 at Uni, a couple of projects have been coming to a close. I thought I'd post some videos of them running.
First up is the above, an interface from the Universities' leaning environment - a load of HTML in an education-targetted CMS named BlackBoard - in to Second Life. Unfortunately, the version of BlackBoard used by the Uni doesn't make any nice data feeds available, so most of the hard work is 'screen scaping' the relevant data out of pages. Because most of the work went in to this, I didn't get to experiment much with new-paradigm UIs - the above is the result of 12 weeks of code (!!).
Each module the student is taking is represented by an orb, which (when clicked) tells the student any new announcements that have been posted to that module.
(click here to open the video in a new window
Above is the fun one - a demo of my crypto accelerator. It's a fairly undramatic demo, but it represents, again, around 12 weeks of work. The accelerator is configured to crypt using XTEA at 32 rounds, and fed with some test data which it encrypts and then decrypts. Pay no attention to the timestamps - this is a fairly early and unoptimised version of the design. Things should be faster later on. |
|
|
| (no subject) |
[Dec. 28th, 2008|04:35 am] |
I recently threw together a small compute cluster from some spare parts I had. It comprises five Pentium 3 chips, clocked at 1Ghz, and one mobile P4 clocked at 1.4Ghz. Each has a relatively small amount of RAM - around the 64MB mark - mainly because I don't have more lying around.
The cluster boots via a modified version of (the rather excellent) LTSP environment. Each node is diskless, and boots via PXE, downloads a kernel/initrd, and mounts root via NFS. Obviously, disk access is slow in this configuration (compounded by the fact that the 'server' machine runs IDE disks, at UDMA 66) but it kicks out relatively little heat and chomps relatively little power. Each node can be powered on via WoL - important since the cluster is stored away from my house, making physical maintainence hard. There's a MOSIX kernel compiled, but not set up properly, as I don't need MOSIX yet. It's a shame that openMosix died (although great that I can get a student license of MOSIX!)
Anyway, one thing that the cluster is configured for is to use distcc, a distributed C compiler. There is some discussion over the optimum number of C programs to compile in parallel (the '-j' flag as passed to 'make') and so I thought I'd run some investigation.
I took a vanilla 2.6.27-10 kernel, ran make defconfig, and then ran make CC=distcc HOSTCC=distcc -j $j, where $j ranged from 1 to 10. The results were interesting, shown below:
These results were taken when the cluster comprised of four PIII 1GHz chips, with around 64 or 128MB memory per node. distcc was configured not to use the server machine as a compilation host (any more than was neccesary).
We can clearly see that 6-8 parallel processes is optimum - taking 15 minutes to compile the kernel. The elevated time at 5 is possibly a freak value, and values above 8 are probably the result of network congestion.
So there you have it - for a five node NFS-root no-swap low-memory p3-1Ghz cluster, use -j 7 :D
Hope this post is useful for someone. Please leave a comment if it is, or comment just if you think I'm wrong! |
|
|
| (no subject) |
[Dec. 20th, 2008|04:08 pm] |
Really silly problem I just spent a good three hours figuring out, just posted here in case anyone else has a similar problem:
Situation was that I was booting diskless clients via TFTP/NFS root (ie, using LTSP). They're debian workstations. In order to get WoL working, I built a new kernel, pruned it (over-enthusiastically) and broke something.
Symtoms were that:
* No TTY open on the machine console
* SSH reports "sshdPRNG is not seeded"
* X didn't start
I scratched my head for ages, verifying that /dev/random and /dev/urandom were present, twiddling .config options, verifying that random.c was being compiled, etc. All the TTY options were present/enabled. Further up the kernel logs was the warning that /dev/mem was not found, which was bizarre, since all that was enabled, too.
Now, I have no idea how - but I had disabled UNIX domain sockets. D'OH! |
|
|
| (no subject) |
[Nov. 26th, 2008|04:10 am] |
So, a glimmer of progress in the saga that is my final year project!
( progress! )
It works, with a 'testing' XOR core. The bottom dump is the key xor the second dump. Yay! That's running in FPGA via pcie.
The project is, looking at my original Gantt chart, around three weeks behind schedule, mainly because I had a bit of a nightmare (well, a lot of a nightmare) grappling with the Xilinx PCI-Express core. Once I got it running - about a week ago - everything else was fairly straightforward. I'm hoping I can make up for lost time over Christmas (which should be do-able, because I built so much 'spare time' in to my schedule, anticipating this kind of catastrophe) and get everything done on time.
Also troubling me is that all my Uni stuff this semester is due on the same day, and there's some impressive juggling going on between my VR group project (to model a chunk of the Univeristy in Second Life) and another module, in which I am required to model some stuff in a formal Finite-State language.
Most of the VR stuff is done now. My part of the project is to interface the 3D environment to the Uni's web-based 'learning environment', Blackboard. Blackboard, however, doesn't kick out any nice webservices, so there's a lot of screen-scraping and generally horrible code going on in there. All of that's now done, though, so I just need to finish it off by writing some scripts in Second Life to make everything feel immersive. Watch this space. |
|
|
| (no subject) |
[Oct. 29th, 2008|10:36 am] |
I've been looking at linux config versioning systems, lately. The idea of having all my configuration files under version control is really appealing to me - not only can I recover really easily if I break one, and view history of things that've changed, but it should also make it really easy to 'back up' a server without having to image the whole system, since I can just snapshot all the configuration files, install the OS on to a new machine, and then dump all the config files.
A quick google, and this linux.com article suggest a source control system named Bazaar, so...
mary:/# apt-get install bzr
mary:/# bzr init /
Next, I use dpkg to pull back a list of all the configuration files on my system, and version them using bzr.
mary:/# bzr add `dpkg-query -W -f='${Conffiles}\n' |sort -u| cut -d " " -f 2`
mary:/# bzr commit -m "Initial import"
Next, we test out that it's working, by moving a vital config file, /etc/fstab:
mary:/# mv /etc/fstab /etc/fstab.backup
mary:/# bzr diff
=== removed file 'etc/fstab'
--- etc/fstab 2008-10-29 12:21:54 +0000
+++ etc/fstab 1970-01-01 00:00:00 +0000
@@ -1,10 +0,0 @@
-# /etc/fstab: static file system information.
-#
-#
-proc /proc proc defaults 0 0
-/dev/sda3 / reiserfs noatime 0 1
-/dev/sda1 /boot reiserfs notail,noatime 0 2
-/dev/sda5 none swap sw 0 0
-/dev/fd0 /media/floppy0 auto rw,user,noauto 0 0
-/dev/sdb1 /mnt/space ext3 noatime 0 0
-/dev/sdc1 /mnt/backup ext3 noatime,ro 0 0
mary:/#
Awesome! It's working! Don't forget to restore the moved file.
mary:/# mv /etc/fstab.backup /etc/fstab
Let's take a look at how this aids us in recovering from a catastrophe. Again, we break /etc/fstab to demonstrate. We overwrite the file, check that mount -a is now broken, and then revert the whole system to previous.
mary:/# echo ooops > /etc/fstab
mary:/# bzr diff
=== modified file 'etc/fstab'
--- etc/fstab 2008-10-29 12:21:54 +0000
+++ etc/fstab 2008-10-29 12:24:40 +0000
@@ -1,10 +1,1 @@
-# /etc/fstab: static file system information.
-#
-#
-proc /proc proc defaults 0 0
-/dev/sda3 / reiserfs noatime 0 1
-/dev/sda1 /boot reiserfs notail,noatime 0 2
-/dev/sda5 none swap sw 0 0
-/dev/fd0 /media/floppy0 auto rw,user,noauto 0 0
-/dev/sdb1 /mnt/space ext3 noatime 0 0
-/dev/sdc1 /mnt/backup ext3 noatime,ro 0 0
+ooops
mary:/# mount -a
mount: mount point does not exist
mary:/# mount
/dev/sda3 on / type reiserfs (rw,noatime)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
procbususb on /proc/bus/usb type usbfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
/dev/sda1 on /boot type reiserfs (rw,noatime,notail)
nfsd on /proc/fs/nfsd type nfsd (rw)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/sdb1 on /mnt/space type ext3 (rw,noatime)
/dev/sdc1 on /mnt/backup type ext3 (ro,noatime)
192.168.30.102:/ on /mnt/eva type nfs (rw,addr=192.168.30.102)
mary:/# bzr revert
mary:/# mount -a
mary:/#
Awesome! Let's demonstrate how to commit a change - simply edit your file, and then bzr commit
mary:/# nano /etc/fstab
mary:/# bzr commit -m "a test commit"
modified etc/fstab
Committed revision 2.
mary:/#
That simple, eh?!
Merging all your config files in to another machine is also pretty easy - bzr includes funcitonality to 'push' to sftp. Read more in their five-minute intro.
I think I'm going to be using this a lot! :D |
|
|
| Time found; bits of documentation uploaded |
[Oct. 27th, 2008|10:05 am] |
I managed to find a couple days spare so I've scribbled some quick documentaion and thrown it on the lavalamp site (http://lavalamp.gamesfairy.co.uk). I've also done some more work on lavalamp, though it needs still more before I can release. I'm really hoping for a pre-christmas release, although I can't promise anything since Uni is soaking up my time (and has to have a higher priority).
I've been working on the 'online cv' site, http://cv.gamesfairy.co.uk. The idea is that I can present it to potential employers, along with my CV, just to let them get a feel for all the 'projects' I do (and to demonstrate initiative - apparently these people like that sort of thing). Any comments on it would be great - any error that's flagged up before I point prospective employers to it is a good thing. I still need to do a load of work to it - like adding more of my University work - which I'm hoping I can get done in the next couple of weeks, before I start applying for jobs. |
|
|
| (no subject) |
[Oct. 25th, 2008|04:36 am] |
Well, I appear to have 'dodged a bullet' with my Final Year Project - when I actually started work on the project, I discovered it was pretty much impossible to do what I wanted to do, in the way I wanted to do it.
While I was aiming to 'hide' my extra encryption/decryption logic in the extra CAS latency cycle gained by using CL2 RAM in an CL3 system, I overlooked the fact that cas latency timings are used only during data writes, and not during reads, as their intention is to allow an error-free comparison of charges in memory to reference values. There's a timing parameter which specifies a minimum time after a write, but that doesn't seem to be widely supported. Because of this, and some other issues (mainly relating to wanting to run 'proper' strong encryption instead of just a simple xor scheme) I had to look in to other approaches.
My 'backup plan' of using 'registered' memory, at the expense of mainstream motherboard compatibility, was rapidly shot down as I realised that only control lines - not the data/address buses - are registered in such a scheme.
I hastily fabricated a new plan - to attatch an FPGA in between the CPU and motherboard of the system, and did a few days research and all-out panic-driven studying. I finally concluded that this was also a bad idea, as it relies heavily on work in areas I'm not confident in (such as high-speed PCB design) and I was eventually forced to drop the whole project idea - two days before I was due to hand in a 'project plan' document detailing the project, and containing a detailed plan of how I was going to perform it.
I did, however, somehow manage to think of a new project, and build this plan. I'm not entirely sure how I achieved this (besides the obvious 'I didn't actually sleep').
The new project is a study in to the ease of design, speed, and power usage of an FPGA-based hardware implimentation of various encryption algorithms. Put simply, I'm going to develop FPGA cores for six or so crypto schemes, benchmark them all in as many different areas as I can (including in a fairly 'real-world' application), in as reproducible a way as possible, and then write a nice report about it.
It's a lot less star-spangled, I think, but it should be orders of magnitude easier than the previous proposition. I don't need to do any of the stuff that I'm really inexperienced at - ie, PCB design - which would normally be a bad thing, but the previous project disaster has really brought home how much I need to be careful not to overstretch myself, and keep the project as do-able as possible. Having said that, there are a few things in there that I haven't done before, which should prove challenging. I've got around 30 weeks to do it, and since I've planned my time really generously, I'm hoping I'll get a couple of weeks spare to 'enhance' the project with extra algorithms, or benchmarks on different hardware.
Wish me luck! |
|
|
| (no subject) |
[Oct. 10th, 2008|05:24 am] |
Today I chose a final year project to work on during the final year of my degree at Bradford University.
My idea is to make a DIMM memory module, utilising an FPGA to encrypt and decrypt data on-the-fly before writing it to/reading it from main DRAM. This is intended to safeguard the 'cold-boot' attacks as researched by various researchers from Princeton University (see
http://citp.princeton.edu/pub/coldboot.pdf and http://en.wikipedia.org/wiki/Cold_boot_attack). Essentially, this attack involves reading insecure data from a computer's DRAM memory after power-off, given physical access.
My proposed project would take reference DIMM designs, from JDEC, and add an FPGA in between the data lines of the system and the DRAM. This would then use an extra CAS-latency cycle in which to perform an XOR encryption (with as long a key as possible, and some clever key-scheduling to prevent known-null-plaintext attacks inherent with XOR encryption). This extra cycle would be obtained by running, for example, CL2 DRAM at CL3 speeds. The memory
would probably be of PC100 single-data-rate or PC133 single-data-rate speed, as I believe that this project with current memory technologies would be much more difficult.
Involved work would include the FPGA design itself, the PCB layout, and a significant amount of testing. Deliverable would be the encrypting DIMM module, and possibly a second sub-project capable of demonstrating its functionality (for example, a board to dump DRAM data to a USB host, as a 'worst case' attacker scenario).
This should be really fun to work on and get my teeth in to over the next year. Unfortunately, though, it does mean that work on lavalamp has to be slowed considerably - I just can't afford to neglect my studies this year. Work on lavalamp has been good over the past couple months, but it isn't quite at the quality I want for a release yet. I was aiming for around two weeks' time, but now this is going to be much extended (unfortunately in a totally unpredictable way) because of Univeristy.
I will, however, leave you will a screenshot of the part of lavalamp I've been working on recently, just in case any of you lose intrest and/or start to think of it as vapourware.
(and for the skeptics, yes, all of that displayed functionality is, er, functional).
Definitely coming along. I've still got a few things to finish - about two weeks worth, if I wasn't at University - but I honestly don't know when I'll get time to finish it. Hopefully not long. |
|
|
| (no subject) |
[Sep. 2nd, 2008|05:39 pm] |
Yesterday, I decided to set up my SCSI disk subsystem in my main machine again, retiring the old SATA disk I threw in there as a 'temporary measure' which ended up being fairly permanent.
My setup consists of two 18GB, u320, 15k spindle speed drives, and a newly-purchased PERC4 pci-express RAID controller. IMO, it is worth sacrificing the insanely-fast seek time of a single 15k drive for the slightly-slower-but-still-very-fast seek time of a striped array of two 15k drives.
Unfortunately, there are a few issues with this idea.
Firstly, the PERC4 simply doesn't work in my board, or my other pci-express board. Googling about, it seems that these cards are pretty picky, and some people claim that the best way to make them work is to place them in a dell machine, and flash them to the firmware of the LSI megaraid 320-2X, which is apparently the same hardware. I don't have any new dell hardware, but a friend of mine does (thanks Mat!) who has offered to flash the firmware for me, but I'm wondering if I'm happy with depending on such a flaky piece of hardware.
Secondly, one of my 18GB 15k drives has died a death, so at the very least, I'd have to replace it with another 18. I'd be happier buying a single 36 (or even 72) since they've come down in price now, and keeping everything on one disk will give me slightly faster seek times (albeit at the cost of slightly less throughput).
These two things make me very tempted to switch to a totally different subsystem. The way I see it, I've got a few options:
* Hope PERC4 runs in PICE, buy bigger 15k SCSI disk (or possibly two and mirror them).
* Fish in my spares box, pull out a PCIX (not pcie) fakeraid card, set up single drive or striped array, and just live with the PCI bottleneck (at 32/33mhz=133mb/sec).
* Attach an enthusiast-level drive to the onboard SATA controller. This looks like a Raptor or the like, which are apparently blazing drives, despite ahving only a 10k spindle speed. The onboard controller won't do any caching, though, being a weedy little thing, although it should be pretty fast to talk to the rest of the system.
* Sell the PERC4, and buy an PERC5 SAS controller. Whack 512MB or so RAM in it, attach some server-grade super ultra mega fast drives (I'm thinking if the Savvio 2.5" drives) and do it properly.
* Sell the PERC4, say 'screw it', and buy a 32GB SSD. Don't bother with an expensive controller, since it's so stupidly fast to transfer data that caching it isn't as big an issue. Consider buying a proper controller later, when I'm rich, to avoid any CPU hit from my onboard SATA hardware.
My main areas of interest are c# development (compiling from VS, Intellisense, etc) and FPGA synthesis using Xilinx ISE. I think the faster seek time and big cache of a 'proper' controller will really benefit me when I do FPGA stuff.
Any storage gurus reading this blog want to comment? |
|
|
| navigation |
| [ |
viewing |
| |
most recent entries |
] |
| [ |
go |
| |
earlier |
] |
| |
|
|
![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small}
gamesfairy's journal