Welcome to my blog!

I post here about my technical ramblings, usually on the topic of computer security and other fun stuff. Comments are enabled, and trolling will result in a swift deletion, but other than that, I'm more than happy to answer any questions that people have on any of the entries.

An RSS feed is available here for your bookmarking pleasure.


This journal is exclusively for 'technical' posts. Anyone who comments in it discussing my personal life will be speedily deleted. Comments on my personal life should go in my personal journal.

Enjoy!

-Alan/Aliz/Randomdude/Gamesfairy (goddamnit where do I get so many names)

So. Another post about the security scanner.

I've spent (most of) the last two days playing, in some way or another, with memory breakpoints.

The scanner requires that it be able to set breakpoints on memory accesses. It needs to be able to say "Tell me when memory at XXXX is accessed" and run the target code otherwise unsupervised. This is a common feature in debuggers - so common that Intel added the feature in hardware.

So, the CPU itself is capable of performing memory breakpointing. However, it has a couple of downsides - firstly, we're limited to four memory breakpoints, and secondly, the CPU only tells us _after_ memory has been accessed, not before. So, for example, if we set a memory breakpoint at some location and the program writes to it, there's no way of getting the data back.

OK. So we can't use hardware breakpoints. But that's no real biggie - we can just use the software breakpoints that the debugger provides (the security scanner operates in parallel with a well-known debugger package). So I went about doing this, and after a lot of headscratching, realised that - bizarrely - the debugger package I use notifies me _after_ memory accesses too. Odd.
I figure, OK. Fair enough. I'll just write my own memory-breakpointing system. All I have to do is mark pages as guard pages, and catch the exception that gets thrown when that page is accessed. An exception will be thrown for every hit on the _page_, so I'll filter them by address. It'll be slow, but it's cool. No problem. So I implement that.

And it works. Oddly, though, the exception info I get passed by the debugger looks like random crap from the heap. No amount of playing about will get a sensible result. Unfortunately, I kinda need that information - one thing it contains is the memory address which was hit. I play with it for a bit more (the debugger I'm building around is a bit finnicky) and still can't coax anything out.

Eventually I find a line in a changelog of v5.2 of the software. "BUGFIX: SDK: dbg_exception event parameters were wrong". I'm using 4.8. The reason I'm using 4.8 is because there's a not-for-profit version of 4.8 available free. 5.2 costs around 300 GBP.

I eventually cave. I'll pirate a version of 5.2. So I do. And it doesn't co-operate with my plugin at all. Things just don't work. I realise that there's going to be a lot of time spent effectively porting it to the newer version of the debugger, before I even get to see if my problem is fixed.

I spend a lot of time trying to coax the debugger in to doing what I want to do. Perhaps it's time to ditch it for another?

It'd be an amount of work to adapt the scanner to use GDB instead, but GDB has a couple advantages:

* Free-as-in-money to use
* Dead easy to replace with scripts, in order to test the scanner's response to various debugger behaviours (a BIG THING)

I'm loathed to sink more dev time in to doing _more_ stuff, but I think this is the best way forward. Better tests would make me happy.

Got the caching layer for the security scanner written last night.

As the scanner runs, it produces ASCII files which are sent to a field solver (CVC3 in this case). Obviously, there's a bit of a performance hit in producing these ASCII files, as opposed to just passing a data structure, but it makes caching and load balancing just so much easier that I decided to go with the ASCII approach.

Some of the queries are identical, especially when repeatedly running in 'verification mode'. Since I want to set up a continuous build system, and since it'd benefit performance anyway, it makes sense to cache the ASCII solver input and the ASCII solver output and respond to repeated queries.

This was fairly straightforward to implement. Since there's already a 'co-ordinator' which runs centrally, and which talks to all the instances of the actual debugger, it makes sense to throw the caching code in there. I decided to use a full-blown database to store my data, since I think that's faster (in terms of dev time) than re-inventing the binary search, and about the same speed (run time) in actually running. If anyone can prove me wrong, go ahead :D

Anyway. After a few runs of the same code - pretty much ideal conditions for the cache - I've ended up with ~4000 rows in the DB, and 2112 hits to 856 misses (71~ hits). Each hit takes, on average, 87ms, and each miss 226ms. I've made a net time saving of 137ms per hit, or 293569 - almost four minutes - over the whole run. Yay!

So, work churns on in the topic of the scanner.

I've just been doing a 'verification run', in which the program is run and the scanner observes each instruction, simulating it in terms of dataflow, and then compares results.

It's falling over at the moment, due to an inability to handle a particular instruction in 16bit mode (I'll fix that in a mo). But before it does so, it simulates 250 (non-unique) instructions mainly in the C runtime library, using a variety of addressing forms, and testing my modr/m and SIB decoding, and instruction coverage. (in addition to that 250, there are 35 occurances of around 6 unsupported-at-the-moment opcodes, mainly because certain CPU flags aren't implemented yet (ie, it's not worth writing the 'ADd with Carry' opcode until the carry bit works) or because they reference memory by segment (which isn't supported just yet).

I have, of the 256-ish[1] single-byte opcodes, 67 (26%) implemented.

The 'API call mocking' system is running. When an app calls, for example, Recv(), the call isn't executed, but trapped and symbolic data returned. Other calls which are difficult to verify are similarly modified (such as GetTickCount, which returns different data on each call).

Next big functionality:

* More instructions, until I can run all of my VS-compiled C++ test app (including most of the C runtime library)

* More API call mocks

* Database-driven caching for solver files (yay)

* Minor modifications to allow parallelisation (the app is architected to parallelise really well, but this was dropped a couple months ago to speed up dev)

* Add logic to actually find holes ("... if EIP can be equal to 0x41414141..")

* Extend variable-length buffer support with some ideas I had in the shower yesterday

* (maybe) speed up constraint solving with some other ideas I've been mulling over

* Fix memory leaks.



Still hoping to have the app finding holes (preferably generating exploits) in my example 'vuln' simple stack-based buffer overflow code by the new year. This'll depend, though, on how much time gets dedicated to the scanner over other things I have to juggle.

[1] there are 256 opcodes - 0x00 through 0xff - but not all are used, some won't be supported (MMX or SSE or whatever) and some opcodes are escapes to much more functionality.

So! You could be forgiven for thinking that the Lavalamp project is dead!

The website (along with two others I own) was unfortunately lost during the great VAServ hack, and up until recently, I have been simply too busy with University to redesign it. However, the time has come that I have some free time - so hey presto! A new site. As the more astute may be able to tell, web design is really Not My Thing, so please do feel free to point out any glaring errors or horrible face-burning design issues I've stumbled upon!

The project itself is coming on well. As outlined in my previous postings, there's a lot I want to do with Lavalamp, but precious few moments in which to do it. I am, however, planning (at the moment) to have some working code released within a couple of months (of October). However, since I've been promising a release since, ooh, about two years ago, you probably don't want to hold your breath!

I've stuck some screenshots in the 'user documentation' section that should show how things are running. The 'rule' mechanism is shaping up fairly well, and looks like it'll be quite powerful if I ever manage to finish it. Lets hope, eh!

Man. Coding this security scanner - in C++ - is really reminding me how much more enjoyable things are, the 'closer to the metal'. It's also, thankfully, giving me a great insight in to how the 'upper layers' work - mainly C#-type stuff. My C++ has always been terrible, although my C was usually fairly strong, so I had a lot of trouble previously with working out how c# is implemented by the compiler - but now, knowledge gained in C++ is helping to fill the gap.

Anyway. Progress on the security scanner.

I've finished implementing the 8bit memory layer, and have adapted a couple of the x86 instruction handlers to use it. I've got decent tests for the ones I've adapted - three instructions in total - which test thigns like the values received and how dependencies are mapped between the variables. Unfortunately, each is quite substantial, in terms of code, so it looks like it'll be prohibitive to write tests for every single instruction (especially since there's a gazillion of them in x86).

There's another 'verification mode', in which the scanner traces instructions with IDA and verifies that it gets the same answers, but that's pretty useless for dependency tracking. Perhaps I can use this mode, and investigate bugs that it flags using per-instruction tests.

The 8bit memory layer took longer to write than I'd hoped, coming in at arrounnnddd 4 or 5 days. While doing that, though, I've cleaned up a load of old code, and re-organised to make things more OO. I was also called to rewrite much of a solver function, so while I was at it, I implemented functionality that was on my waiting list, estimated to take a day or two, so that can't be bad.

The functionality I talk about is variable re-labelling - when I write a file to send to the solver, I label each variable in order of appearance in this file, instead of with its per-run 'name'. This'll mean that I can cache things a lot more easily and effectively (and yes, caching is still on my to-do list).

The next steps are to implement emulation of lots more instructions, make a few tweaks here and there, finish a bit of functionality, and it should get to a usable state. That shouldn't, hopefully, take more than a month (barring huge gaps in functionality which I've forgotten about).

Here's hoping, eh? :D

I've mainly been working on the security scanner over the past couple of weeks, when I've been able to find time. I've been very busy moving house, and doing other IRL stuff.

Anyway.

I think I mentioned earlier, that the security scanner models memory as a set of discrete 32-bit locations, and how much headache it's caused me. I've mostly been 'converting' to an entirely 8-bit memory model, and using 32-bit operations purely as constructs to model the interaction of various memory locations. For example:

4 bytes of memory at 0x00 thorough 0x04 are added to memory at 0x10-0x14, using a 32bit operation.

The scanner would create 8 entries in the 'memory to variable' map:

memory at 0x00 is represented by variable A
memory at 0x01 is represented by variable B
memory at 0x02 is represented by variable C
memory at 0x03 is represented by variable D
memory at 0x10 is represented by variable E
memory at 0x11 is represented by variable F
memory at 0x12 is represented by variable G
memory at 0x13 is represented by variable H

and would then create a 32-bit 'helper' value X. Next, the scanner marks that A through D are dependent on X, so you have:

AB CD
| | | |
VV_VV___
| X (32bit )|

sorry about the ASCII art!

Similar is done for a new var, Y, and E through H:

E F G H
| | | |
V_V_V_V__
| Y (32bit )|

The scanner then defines that Z is equal to the result of a 32bit add, X+Y.

Z = X + Y

To store computer data in memory, we can't simply set EFGH - what if something else depends on them (ie, before this add, what if a different operation used EFGH)? We make new 8bit variables, IJKL, and assign them to Z. We then tell the system that memory at 0x10 through 0x14 correlates to IJKL.

The final outcome of the system:

AB CD
| | | |
VV_VV___
| X (32bit )|

EF GH
| | | |
VV_VV___
| Y (32bit )|

Z = X + Y

I J K L
| | | |
VV_VV___
| Z (32bit )|

memory at 0x00 is represented by variable A
memory at 0x01 is represented by variable B
memory at 0x02 is represented by variable C
memory at 0x03 is represented by variable D
memory at 0x10 is represented by variable I
memory at 0x11 is represented by variable J
memory at 0x12 is represented by variable K
memory at 0x13 is represented by variable L

This is then serialised in to an input file for the STP solver I use - cvc3 - which comes out like this (as an example, we say that ABCD is hardcoded to 11223344 and CDEF is hardcoded to 55667788):

(snip uninteresting bits)
ASSERT( X = A:B:C:D );
ASSERT( Y = E:F:G:H );
ASSERT( Z = X + Y );
ASSERT( A = 0x11 );
ASSERT( B = 0x22 );
ASSERT( C = 0x33 );
ASSERT( D = 0x44 );
ASSERT( E = 0x55 );
ASSERT( F = 0x66 );
ASSERT( G = 0x77 );
ASSERT( H = 0x88 );
TRANSFORM(Z) ;

Obviously, I've edited that code to make it clearer. It denotes a series of 'simultaneous' assertions - the solver will find an answer which satisfies all the assertations. The final line - TRANSFORM- asks the solver to tell the user a value of Z which does this. The only value it can come out with, for this particular STP file, is 0x6688AACC - which is 0x11223344 + 0x55667788.

Notice that 'memory' has been totally abstracted away, here, in favor of 'variables'. Modelling memory is bad, mmmkay. We're running at so low a level - the machine code level - that heap corruption will be modeled just as if it were a legal situation.

This example shows how easy it is to then add other nnon-32-bit constraints. You could then add

ASSERT( F = N OR P );
ASSERT( N = 0x60 );
ASSERT( P = 0x06 );

and get the same result.

The current codebase differentiates between a register - reg_32_t - and four bytes of memory. This is bad, because we can't treat the two identically. Especially in x86.
x86 has a concept called the 'modr/m byte', which is a byte tacked on to most instructions specifying what to operate on. It can specify a register (well, strictly speaking, two, actually, target and source) or a memory location (or a deref'fed pointer, but that's almost the same thing, as far as we're concerned). Having a function to decode this byte was dead easy when memory was modelled as a series of registers - it just returned a 'register' stru- er, object - and the code did with it whatever it liked.

It looks like, to return to this nice neat state of affairs, I'm going to model registers as groups of 8bit discrete registers. I'm a bit worried this is going to be dog-slow, but it's the neatest (only?) way I can think of doing things.

I really miss c#.. this project is reminding me how inexperienced I am at C++, and OO in general. If I was to restart this project from scratch, one thing I'd do would be to OO-erise it a lot more.

Apologies for the hard-to-read nature of this post. It was a tossup between ascii art and just scanning my 'working out' notepad.. And its midday and I'm tired, having been coding all night. Comment, or ask me via IM, if there's anything I should say more about.

* Been playing some more with Opsview. First step was getting its 'agent' program running on Jaunty. I did a few things to get it going:

1) I just downloaded the latest version (https://downloads.opsera.com/opsview/yum/3.1/centos/5/i386/opsview-agent-3.1.0.2553-1.ct5.i386.rpm)

2) Turned it in to a dpkg it with alien, and installed it
alien -d opsview-agent-3.1.0.2553-1.ct5.i386.rpm
dpkg -i opsview-agent-3.1.0.2553-1.ct5.i386.rpm

3) Added the 'nagios' user, chgrp'ed the nagios dirctory, and symlinked from the openssl libs to libssl.so.6, which is (another) dirty hack, but peh. :D
useradd nagios
chgrp -R nagios /usr/local/nagios/
ln -s /usr/lib/libssl.so.0.9.8 /usr/lib/libssl.so.6
ln -s /usr/lib/libcrypto.so.0.9.8 /usr/lib/libcrypto.so.6

Kapow! It worked (for me).

* Need to install NTP to sych times.

* After this I installed snmpd on the machines I'm monitoring, and scribbled up some simple scripts to monitor SMART information. First off, I installed snmpd and got that working, then installed smartmonctl, and checked that I could get data from all my drives. Success! I went ahead and wrote a dead simple one-line script to get drive info, and parse it (improvements are welcome, bash scripting is not my forté!):

root@anna:/home/smb-aliz# cat /usr/bin/smart-drivetemp
#!/bin/bash
exit `smartctl $1 -a | grep Temperature_Celsius | cut -b 85-90`
root@anna:/home/smb-aliz# smart-drivetemp /dev/sda
root@anna:/home/smb-aliz# echo $?
42


Ace. So now we've got a script that accepts a drive name, and returns the drives temperature. Now to wire it up to snmpd. The only problem I ran in to here is that the SMART tool needs to be run as root, for obvious reasons. snmpd runs as the snmp user, so I decided to make smartctl suid root (chmod u+s /usr/sbin/smartctl). I could've made the script suid, but I'm not quite that silly. Obviously, it's a bad idea to make anything suid root, and an even worse idea to make a tool which can take your drives offline suid root. Don't do this on important machines which evil hax0rs could get an account on, kids. If there's a better way than setting smartctl to be SUID root, please let me know..

So. After doing this, we tell snmpd to run it:

echo exec smart-sda "/bin/sh /usr/bin/smart-drivetemp /dev/sda" >> /etc/snmpd.conf


And we check it's working.

root@anna:/home/smb-aliz# snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.8
UCD-SNMP-MIB::extIndex.1 = INTEGER: 1
UCD-SNMP-MIB::extNames.1 = STRING: smart-sda
UCD-SNMP-MIB::extCommand.1 = STRING: /bin/sh /usr/bin/smart-drivetemp /dev/sda
UCD-SNMP-MIB::extResult.1 = INTEGER: 42
UCD-SNMP-MIB::extOutput.1 = STRING:
UCD-SNMP-MIB::extErrFix.1 = INTEGER: noError(0)
UCD-SNMP-MIB::extErrFixCmd.1 = STRING:
root@anna:/home/smb-aliz#


Awesome. Now opsview can give me nice graphs of my drive temperatures! And can SMS me if they get too hot! Of course, the possibilites are endless. It's a good idea to also wire up alerts on other SMART properties - Seek_Error_Rate and Raw_Read_Error_Rate spring to mind - and it's an even better idea to wire an alert up to the general SMART health status (smartctl -H/dev/sda). Happy monitoring!

matthewrwright recently recommended Opsview to me as a network monitoring tool. It's built on Nagios but is apparently a lot easier to configure.

Unfortunately, though, packages are only provided for debian Etch, Lenny, and Hardy. this post seems to indicate that there are no plans to make a Ubuntu Jaunty package available (although the packages seem to work with previous versions of Ubuntu). Nevertheless, I've gotten opsview working on my Jaunty box, so here's the way I did it:

0.5) Understand that this is completely without warranty, and may break your apt configuration, your machine, your network, etc. I did this on a virgin jaunty box - I make no statements that it'll work for you. Don't forget to backup first!

1) Install the 'equivs' package.
apt-get install equivs

2) Create a dummy package, purporting to provide libltdl3.
equivs-control libltdl3
Edit the created libltdl3 file, changing 'package' and 'proves' lines to contain 'libltdl3', like this:
Package: libltdl3
Provides: libltdl3
Now, create the package:
equivs-build libltd3

3) Install the dummy package:
dpkg --install libltdl3_1.0_all.deb

4) This step is a bit of a hack, but works so far (again, don't blame me if it breaks stuff..)
ln -s /usr/lib/libltdl.so.7 /usr/lib/libltdl.so.3

4) Add opsview repo to sources.list, and install it.
apt-get install opsview

If you don't feel like doing steps 1 through 3, you can simply download the deb package I made and install it. You'll still need to symlink libltdl, though.

Since last time, on channel gamesfairy:

* Talked to a couple of friends - one of whom works for BT - about telecoms. Really reminiscent about all the stufff I used to do with telecom (well sort of).

* Saw the slashdot article about the dude that ran his own telco carrier. Interested.

* Remembered the recent iphone stuff, apparently obtianed by fuzzing the iphone through some 'custom' stuff. IDK how he did it - I'd probably have emulated it, if it wasn't too much trouble? IDK.

* I'd really like to play with this. Run a femtocell, hooked up to one of my phones (and with too little power to get out of my house) and send the phone bizarre shit until it explodes. I should add that to the to-do list.

* I wonder how hackable the commercially available femtocells are. Obviously, the best idea would be to use something like the USRP2 and GNU radio (software radio that does 'DC to 5ghz'), which would all fit together nciely and be awesome, but the USRP2 is far too much for me ($1400US + transciever boards). I really don't want to start fuzzing the telco system live, in case I break it.. maybe I'll have An Idea about this stuff later on. Perhaps I could remove the phone's PHY and hook it to an FPGA and then to the PC.

* Might be easier to just emulate the phone - particularly with stuff like the iPhone where a lot of the hardware is well-observed. If I can emulate it (or probe it live, ICE style) I should be able to hook it to the l33t security scanner I'm (still) writing.

* Need to rewrite a chunk of the scanner. I (rather short-sightedly) modelled memory as a set of 32bit integers, under belief that I could just mask in 16- and 8-bit memory accesses. I kinda could, but I get in to big trouble when I look at 32bit operations which span two 32bit values in memory - (eg: I'll have two memory locations, 0x00-0x03 and 0x04-0x07, and the target will request 0x01-0x04). Going to rewrite the memory access layer as dealin gin 8bit values, and tweak it a bit so it becomes easier to write tests for the instructions, too.

* Really on a telecom bender at the mo. I bet there's cash in hax0ring telco kit, too.. IDK how I'd go about selling stuff I found (to the telco, obv, my hat isn't black enough to sell natino-cripping bugs to The Terrorists). I guess if nothing else it'd earn me skills and rep in the Industry, both of which I seem to be lacking.

* Still pondering post-grad career options, if my current job plan doesn't work out. I think postgrad education might be a good bet. I'd like to go 'freelance pentesting' but realistically, I've got no chance in the market (not enough people skills, tech skills, etc). Maybe in a few years.

* Been working on lavalamp. Looks like my plan to scale it back and get a functinal release out should go ahead.