Mon, 16 Nov 2009
This Can't Be Right.
if (eocd[i ] == 0x50 && eocd[i+1] == 0x4b &&
eocd[i+2] == 0x05 && eocd[i+1] == 0x06) {
// if the sequence $50 $4b $05 $06 appears anywhere after
// the real one, minzip will find the later (wrong) one,
// which could be exploitable. Fail verification if
// this sequence occurs anywhere after the real one.
A friend who is poking around at Android code came across that and pointed it out to me. I took a look at the git repository and it's still there (git://android.git.kernel.org/platform/bootable/recovery). I don't know any reality where that will ever be true.
posted at: 15:42 | tags: android, typo | path: /entries/geek | permanent link to this entry
Fri, 18 Sep 2009
Adventures in Soekris Land - Part I
As I've mentioned before I was recently given a net4501 by Jordan. Now that I'm moved into my new house I'm in the process of getting FreeBSD up and running on it. Since Jordan said he was seeing performance issues with it I figured it couldn't hurt to update the BIOS on the device first before I boot FreeBSD on it. It's actually a pretty simple process. All you need is lrzsz (ports/comms/lrzsz) and a serial cable.
Connect your serial cable and power on the board. You'll see the inital POST and then you have 5 seconds to hit Ctrl-P to break out of the boot sequence and into the "Monitor" which is basically a firmware prompt. Once you've got there issue the "download" command. Once you do that you have 30 seconds to start a file transfer of the image you want to flash. Using cu(1) to connect to the board you can use the ~C sequence to fork a child process where file descriptor 0 is the remote tty input, file descriptor 1 is the remote tty output and file descriptor 2 is the local tty stderr. The command you want to run is lsz -X <img.bin>. This will send the image over the serial port to the board which is waiting for it (and stores it at a special location in memory). Once the download is finished you can flash the freshly downloaded image onto the board using the "flashupdate" command.
Soekris has a great manual and a wiki which are both great sources of information.
Now that my board has the newest BIOS I'll be working on getting a FreeBSD image running on it. I think I'll start off with a simple PXE booting environment and eventually move to using the compact flash card as a boot environment. You can expect more updates as my Adventures in Soekris Land continues but for now I need to go to bed - my lawn needs to be cut in the morning (oh, the joys of being a home owner =b).
posted at: 22:27 | tags: soekris | path: /entries/geek | permanent link to this entry
Wed, 09 Sep 2009
Way to respect TTL, VMware.
Here's something fun. Here's a DNS query that looks right:
wxs@ack wxs % dig @4.2.2.1 +nocmd +nocomments +noquestion +nostats syn.atarininja.org syn.atarininja.org. 1771 IN A 129.21.60.158 wxs@ack wxs %
Here's the same query but from a NAT'ed VM (VMware Workstation) using the NAT device as a DNS server:
wxs@rst wxs % dig @192.168.2.2 +nocmd +nocomments +noquestion +nostats syn.atarininja.org syn.atarininja.org. 5 IN A 129.21.60.158 wxs@rst wxs %
The TTL is always 5. It's not like it starts at 5 and counts down. It's always 5. The NAT device acts as a DNS proxy, forwarding requests to whatever DNS server the host machine uses. I checked these requests and the response coming back has the real TTL. Why does VMware feel the need to manipulate TTL values on DNS responses?
posted at: 12:33 | tags: DNS | path: /entries/geek | permanent link to this entry
Mon, 08 Jun 2009
ZFS + NFS = Crash :(
I started to experience a crash recently that was triggered when building something in my tinderbox setup. This particular tinderbox is running on ZFS and uses NFS mounts on localhost. The panic and backtrace look like this:
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0x6dc
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80572e7f
stack pointer = 0x28:0xffffff803e722530
frame pointer = 0x28:0xffffff803e722550
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 1030 (nfsd: service)
[thread pid 1030 tid 100140 ]
Stopped at prison_priv_check+0xff: movl 0x6dc(%rsi),%eax
db> bt
Tracing pid 1030 tid 100140 td 0xffffff00029ea000
prison_priv_check() at prison_priv_check+0xff
priv_check_cred() at priv_check_cred+0x4c
secpolicy_vnode_access() at secpolicy_vnode_access+0x28
zfs_zaccess() at zfs_zaccess+0x1d5
zfs_freebsd_access() at zfs_freebsd_access+0xd0
VOP_ACCESS_APV() at VOP_ACCESS_APV+0x44
nfsrv_access() at nfsrv_access+0xf3
nfsrv3_access() at nfsrv3_access+0x386
nfssvc_program() at nfssvc_program+0x1fb
svc_run_internal() at svc_run_internal+0x6d2
svc_thread_start() at svc_thread_start+0xb
fork_exit() at fork_exit+0x118
fork_trampoline() at fork_trampoline+0xe
--- trap 0xc, rip = 0x8006a0e1c, rsp = 0x7fffffffe6d8, rbp = 0x5 ---
db>
Here's the relevant pieces of prison_priv_check():
/*
* Check with permission for a specific privilege is granted within jail. We
* have a specific list of accepted privileges; the rest are denied.
*/
int
prison_priv_check(struct ucred *cred, int priv)
{
if (!jailed(cred))
return (0);
switch (priv) {
...
/*
* Depending on the global setting, allow privilege of
* mounting/unmounting file systems.
*/
case PRIV_VFS_MOUNT:
case PRIV_VFS_UNMOUNT:
case PRIV_VFS_MOUNT_NONUSER:
case PRIV_VFS_MOUNT_OWNER:
if (cred->cr_prison->pr_allow & PR_ALLOW_MOUNT)
return (0);
else
return (EPERM);
...
Loading up the core in kgdb for analysis it becomes very clear what is going on.
(kgdb) frame 12
#12 0xffffffff80572e7f in prison_priv_check (cred=0xffffff00168f5900, priv=334) at /usr/home/wxs/freebsd/src/head/sys/kern/kern_jail.c:3315
3315 switch (priv) {
(kgdb) p/x *cred
$7 = {cr_ref = 0x1, cr_uid = 0x0, cr_ruid = 0x0, cr_svuid = 0x0,
cr_ngroups = 0x1, cr_groups = {0x0 }, cr_rgid = 0x0,
cr_svgid = 0x0, cr_uidinfo = 0x0, cr_ruidinfo = 0x0, cr_prison = 0x0,
cr_vimage = 0x0, cr_flags = 0x0, cr_pspare = {0x0, 0x0}, cr_label = 0x0,
cr_audit = {ai_auid = 0x0, ai_mask = {am_success = 0x0, am_failure = 0x0},
ai_termid = {at_port = 0x0, at_type = 0x0, at_addr = {0x0, 0x0, 0x0,
0x0}}, ai_asid = 0x0, ai_flags = 0x0}}
(kgdb)
It's clear that cred->cr_prison is bad. But this isn't the real meat of the problem. The first check in prison_priv_check() is to see if we are jailed, and that looks something like this:
/*
* Return 1 if the passed credential is in a jail, otherwise 0.
*/
int
jailed(struct ucred *cred)
{
return (cred->cr_prison != &prison0);
}
Up until fairly recently this function used to contain:
return (cred->cr_prison != NULL);
So, because cred->cr_prison is NULL in our case the check in prison_check_cred() is evaluating to false, when it should be evaluating to true. So now we know why we are crashing (NULL ptr dereference) we still don't know what is causing cr_cred to be NULL.
Credentials like this are derived from a very small number of places. The reason these are wrong is that the RPC code in the kernel doesn't know which credentials to assign when it handles the request. Luckily a workaround has been put in place while a more proper solution is being worked on.
posted at: 12:48 | tags: freebsd, zfs, nfs | path: /entries/geek | permanent link to this entry
Mon, 06 Apr 2009
getrusage() and another example of Linux failure.
So I'm writing some code for work and I need to know the current stack usage. One would think getrusage() would be nice to use. Think again. To quote the manual page on Ubuntu:
struct rusage {
struct timeval ru_utime; /* user time used */
struct timeval ru_stime; /* system time used */
long ru_maxrss; /* maximum resident set size */
long ru_ixrss; /* integral shared memory size */
long ru_idrss; /* integral unshared data size */
long ru_isrss; /* integral unshared stack size */
long ru_minflt; /* page reclaims */
long ru_majflt; /* page faults */
long ru_nswap; /* swaps */
long ru_inblock; /* block input operations */
long ru_oublock; /* block output operations */
long ru_msgsnd; /* messages sent */
long ru_msgrcv; /* messages received */
long ru_nsignals; /* signals received */
long ru_nvcsw; /* voluntary context switches */
long ru_nivcsw; /* involuntary context switches */
};
[...]
The above struct was taken from 4.3BSD Reno. Not all fields are mean-
ingful under Linux. In linux 2.4 only the fields ru_utime, ru_stime,
ru_minflt, and ru_majflt are maintained. Since Linux 2.6, ru_nvcsw and
ru_nivcsw are also maintained.
I can understand having the same struct for portability reasons. It's nice that code can be portable between the two at compile time but this is opening itself up to run-time problems if code assumes all the members of the struct will be populated.
I'm not sure why it's only barely implemented but it's quite annoying. The same function call on FreeBSD gives all the information, and the manual page under FreeBSD has useful information on what each of the members of the struct means. Linux fails, again.
gdb output on Linux:
(gdb) p/x *usage
$1 = {ru_utime = {tv_sec = 0x0, tv_usec = 0xfa0}, ru_stime = {tv_sec = 0x0,
tv_usec = 0xbb83}, ru_maxrss = 0x0, ru_ixrss = 0x0, ru_idrss = 0x0,
ru_isrss = 0x0, ru_minflt = 0x2fe, ru_majflt = 0x0, ru_nswap = 0x0,
ru_inblock = 0x0, ru_oublock = 0x0, ru_msgsnd = 0x0, ru_msgrcv = 0x0,
ru_nsignals = 0x0, ru_nvcsw = 0x7b, ru_nivcsw = 0x1}
(gdb)
gdb output on FreeBSD:
(gdb) p/x *usage
$3 = {ru_utime = {tv_sec = 0x0, tv_usec = 0x0}, ru_stime = {tv_sec = 0x0,
tv_usec = 0x3ac7}, ru_maxrss = 0x8e8, ru_ixrss = 0x3f8, ru_idrss = 0xec,
ru_isrss = 0x100, ru_minflt = 0xeb, ru_majflt = 0x0, ru_nswap = 0x0,
ru_inblock = 0x0, ru_oublock = 0x0, ru_msgsnd = 0x0, ru_msgrcv = 0x0,
ru_nsignals = 0x0, ru_nvcsw = 0xc, ru_nivcsw = 0x1}
(gdb)
posted at: 16:46 | tags: linux, freebsd, code | path: /entries/geek | permanent link to this entry
Fri, 30 Jan 2009
Shmoocon 2009 + New(?) Job
I've been overly quiet the past month or so, and for good reason. I've been herding cats while doing a little bit of work myself to prepare for Shmoocon 2009. I'll be doing security and helping out with Hack or Halo again this year. If you're going to be at the conference stop me and say 'hi.'
In other news I've been unemployed this week, and will be unemployed next week. A while ago I wrote "In a few weeks I'll be starting a new job working on and with FreeBSD. It will be a lot of fun and a huge learning experience for me. I'm certainly looking forward to it, and to sharing all that I can here." While it was a huge learning experience for me it turned out to have nothing at all to do with FreeBSD. I was supposed to be working on using the ports infrastructure (something I know a fair amount about) to help develop a product for Cisco. It turns out that I was a Linux Kernel programmer (something I know very little about). The whole situation was a result of some of the worst timing possible and a heavy dose of silly political situations. At the end of the day I was doing a job I neither wanted nor interviewed for. I consider myself extremely lucky to have such good people at my old job (MITRE) who wanted to hire me back. The end result is that I will be back at a job I like with good people (as opposed to good people at a job I did not like) while still being able to live where I want to live. Considering the tough economic conditions I consider myself extremely lucky to have everything work out the way it did.
posted at: 09:50 | tags: shmoocon, conferences | path: /entries/geek | permanent link to this entry
Tue, 23 Sep 2008
Quick Run Down of How the Freshports Graphs Work.
The graphs I put together for Freshports are actually quite simple. To keep things simple I used jQuery and Flot to do all the heavy lifting for me. If you don't know what jQuery is I'd suggest you go read up on it. I'm certainly no javascript rockstar but the few times I've had to do things in javascript I've always used jQuery and simply love it. Flot is a graphing extension to jQuery. Using these two I was able to generate these graphs in very few lines of code all while off-loading some of the processing power needed to generate them onto the client.
The Setup.
It all starts with graphs2.php. You get both the jQuery and Flot libraries, along with a third file (graphs.js) which is where the meat of the code lives. The important piece to notice is the select element used to select the graph to be displayed. The value arguments to the option elements contain function names we will use later on to do the graphing. These value arguments are all pulled from the freshports database (which isn't how I would have wanted to do things but I had to work within the existing structure of Dan's database).
Further down the page you'll also see some div elements (title, overview, list and holder) which we will act as containers for the various pieces of the graphs we will put together. The last piece to the setup is in graphs.js itself:
$(document).ready(function() {
// Render the first graph.
eval($("select option").val());
$("select").change(function() {
eval($("select option:selected").val());
});
});
This block of code is what is run when the page is ready and should be obvious to anyone who knows jQuery. So that we have a graph to show when the page loads we make our first eval() call: we call the function name in the value argument of the first option element. I'll give more details about how those functions work in The Execution section. The last little bit of code sets up a function to be called whenever the select element changes. Oddly enough we call the function name contained in the value argument of the newly selected option. ;)
The Execution.
So now that you know how the various javascript functions are called let's explore what they actually do. Because each graph is different there is no single function I could write that would handle all the graphs - it would be possible if each graph were the same basic graph. Let's examine the top10committers() function (to be honest, a lot of this graphing code is munged together from the Flot examples):
function top10committers() {
$("#title").html("<h4>Top 10 Committers</h4>");
$("#holder").width(800);
$("#holder").height(500);
$("#overview").hide();
$("#list").hide();
$.getJSON("generate_content.php?ds=top10Committers()", function(d1) {
$.plot($("#holder"), d1,
{
bars: { show: true },
legend: { show: false },
xaxis: {
ticks: function(axis) {
var ret = [];
var i = 0;
for (i = 0; i < d1.length; i++) {
ret.push([i + .5, d1[i].label]);
}
return ret;
}
}
}
);
});
}
We set the html, width and height of the "title" div, and then hide the "overview" and "list" divs (this particular graph does not need those features). The next line is where the real meat of the code lives. We make a jQuery call to get a JSON object from the server. We make this call to the generate_content.php script on the server with a ds (dataset) argument of "top10Committers()". The generate_content.php script executes the necessary SQL, formats the data into a JSON object and returns it. You can see the JSON object yourself. Upon getting a response back from the server the getJSON call executes a function embedded in the call. This function is a standard Flot plot call - $.plot() - so I won't go into the details of it. If you want to read more on how Flot works there are examples, API documentation and a basic README. If you want to explore the other functions being called you can help yourself.
The generate_content.php script is a bit ugly (and luckily not visible to the user) but it is a large switch statement. For any given recognized ds value it will execute a query and generate the JSON object (by hand) to send back to the caller. There is no dynamic SQL queries going on. All of the queries are literal strings passed directly to the SQL engine. In the event that an unrecognized ds is passed to the script it simply returns an error. Flot won't know what to do with this and will refuse to draw the graph.
The Conclusion.
The end result is graphs which can easily be automatically updated by sending a new query to the server. At one point I actually had the graphs updating on a timer. It would be very easy to make these graphs auto-update like that, but I didn't think it would be useful given that a lot of the data is aggregated to a monthly or daily basis. Flot also gives us the ability to interact with the graphs (zooming, selections) with minimal effort. The code for doing this is documented in the Flot documentation so feel free to look there for more information.
The normal approach to Freshports graphs is to generate the static images on a periodic basis and simply serve the image to the browser. This approach means we can have graphs that can be updated with a single SQL query on demand and all the processing power to display the data is off-loaded to the client. There is no need for a periodic script to run to collect the data and generate the graph. It could mean a slightly higher network load as the data being requested can be bigger than the actual graph. For example the Commits Over Time By Committer graph takes a few seconds to load the data (388KB), but once it is loaded the graphing is instantaneous. Personally, I like the ability to graph multiple people at once (and yes, I'm aware of the bug where graphing too many people causes the legend to overflow the container and the graph to look bad).
So I don't know if it's a better way to generate graphs but they certainly are pretty, easy to do, and fun.
posted at: 09:23 | tags: freshports, visualizations, geek | path: /entries/geek | permanent link to this entry
Mon, 22 Sep 2008
Freshports Graphs Are in Beta
Remember those graphs I cryptically wrote about twice? Well, they just got committed to beta.freshports.org, and will eventually make it into production I think.
I've also been asked to write up an entry about them for news.freshports.org but I suspect that will have to wait till tomorrow night. I'll likely just announce them over there and then follow up with a post here about the details of how they work. It's actually quite simple, thanks to jQuery and flot.
posted at: 22:30 | tags: visualizations | path: /entries/geek | permanent link to this entry
Sun, 10 Aug 2008
Freshports Graphs
Remember those graphs I was talking about a while back? Well, I finished them well enough to show off to people back then. In the interim I was asked if I wanted to prepare a patch to include them in Freshports. After traveling a bit, working on other things, and generally having a life I finally got around to doing that (for the record, I've only had Freshports CVS access for a week). I even fixed up some bugs in them. The patch is a single line addition, 4 new files and a handful of SQL. I'll be sending that off tomorrow night and Dan will hopefully approve of it. Once the graphs are online I'll be sure to post a link here. It's fairly easy to generate new graphs so I'm open to doing that if people have suggestions once they see what is currently available.
posted at: 23:02 | tags: geek, visualizations | path: /entries/geek | permanent link to this entry
Thu, 07 Aug 2008
Stupid make.conf Tricks.
No, I'm not dead. I was on vacation last week for my honeymoon and a friends wedding in Hawaii. I'm back now and trying to get my life back on track, but I'm not sure how well that will work since I'm moving to North Carolina in a month or so. I figured I would get back into the swing of things by posting a question I was recently asked and then give you my solution to it. It's a simple one to start things off again.
I was asked how you would go about removing a specific CFLAGS setting for a specific port. This is slightly different from the more common case: setting specific CFLAGS. The more common case is solved with:
.if ${.CURDIR:M*/foo/bar}
CFLAGS=baz
.endif
That is only one way to do it. There are others including things like portconf. The answer to the question I was asked is:
wxs@ack /usr/ports/devel/git % cat /etc/make.conf
x="foo bar baz"
.if ${.CURDIR:M*/devel/git}
x:=${x:C/bar //}
.endif
wxs@ack /usr/ports/devel/git % make -V x
"foo baz"
wxs@ack /usr/ports/devel/git %
I also have another answer, which I think is a cleaner solution (it's not the same as the one above but it illustrates the point):
wxs@ack ~ % cat Makefile
a=foo bar baz
all:
@echo ${a:Nbar}
wxs@ack ~ % make
foo baz
wxs@ack ~ %
posted at: 21:02 | tags: freebsd, ports | path: /entries/geek | permanent link to this entry
