libc's webstation

define_method and ruby 1.8

2009-09-20T00:00:00+04:00

The following code is not thread-safe (in Ruby 1.8):

def quote_string(s)
  self.class.instance_eval do
    define_method(:quote_string) do |s|
      sleep 0.1
      s
    end
  end
  quote_string(s)
end

The reason is block argument s is bound to method’s argument. When new method is called all threads share the same variable, so if you call this method in several threads, it wouldn’t return argument anymore.

Test script is here. When it’s run, it outputs

ruby 1.8.8dev (2009-09-14) [i386-darwin10.0.0]
1 fail [“another test”, “another test”]
2 fail [“another test”, “test”]

ActiveRecord has the following code

# Quotes strings for use in SQL input in the postgres driver for better performance.
def quote_string(original_value) #:nodoc:
   if @connection.respond_to?(:escape)
     self.class.instance_eval do
       define_method(:quote_string) do |s|
         @connection.escape(s)
       end
     end
   elsif PGconn.respond_to?(:escape)
     self.class.instance_eval do
       define_method(:quote_string) do |s|
         PGconn.escape(s)
       end
     end
   else
     # There are some incorrectly compiled postgres drivers out there
     # that don’t define PGconn.escape.
     self.class.instance_eval do
       remove_method(:quote_string)
     end
   end
   quote_string(original_value)
end

So if you use ActiveRecord with postgres and ruby 1.8, make sure do not use threads ;-)

What's wrong with lambda and yield method?

2009-03-03T00:00:00+03:00

What’s wrong with lambda and yield?

def call_block(&c)
  c.call
end

def yield_block(&c)
  yield
end

def lambda_and_call
  c = lambda { return 1 }
  call_block(&c) + 1
end

def lambda_and_yield
  c = lambda { return 1 }
  yield_block(&c) + 1
end

def proc_new_and_call
  c = Proc.new { return 1 }
  call_block(&c) + 1
end

def proc_new_and_yield
  c = Proc.new { return 1 }
  yield_block(&c) + 1
end

[:lambda_and_call, :lambda_and_yield, :proc_new_and_call, :proc_new_and_yield].each {|m| puts “#{m}: #{send(m)}”}

Output:

lambda_and_call: 2
lambda_and_yield: 1
proc_new_and_call: 1
proc_new_and_yield: 1

As you can see, lambda and yield behave strangely.

Hpricot on FreeBSD

2009-02-26T00:00:00+03:00

Sometimes I wonder if FreeBSD hates me… It’s that case. FreeBSD guys have done something to ruby.

irb(main):001:0> require ‘rubygems’
=> true
irb(main):002:0> require ‘hpricot’
=> true
irb(main):003:0> Hpricot(’ <dd> <dl> <ul> <li> ’*200 + ’ </li> </ul> </dl< </dd> ’*200)
Illegal instruction: 4

I can successfully run this code on Mac OS X and Linux, but it fails on every FreeBSD I tried. Technically it’s not hpricot’s bug. If you also have this issue you can try my hackaround for hpricot 0.6.164. It’s a bit slower than original version, but doesn’t kills ruby on FreeBSD.

UPDATE: It’s still doesn’t work… Hpricot parses ok, but .at fails with the same stack overflow.

UPDATE: Finally, I found the solution:

curl ftp://ftp.ruby-lang.org:21//pub/ruby/1.8/ruby-1.8.6-p287.tar.bz2 -o ruby-1.8.6-p287.tar.bz2 && \
curl http://de.mirror.rubyforge.org/rubygems/rubygems-1.3.1.tgz -o rubygems-1.3.1.tgz && \
tar -jxvf ruby-1.8.6-p287.tar.bz2 && \
tar -zxvf rubygems-1.3.1.tgz && \
cd ruby-1.8.6-p287 && \
./configure —prefix=/usr/local/ruby186 —with-iconv-dir=/usr/local && \
make all install && \
cd .. && \
cd rubygems-1.3.1 && \
/usr/local/ruby186/bin/ruby setup.rb && \
/usr/local/ruby186/bin/gem install hpricot

It’s the second time when freebsd ports installs me software with bad patches… Anybody, please consider do not installing freebsd… Let it die

Inspecting dead ruby process with gdb

2009-02-23T00:00:00+03:00

Maybe it will be useful for someone.

What do I have?

A beautiful ruby.core file from freshly died ruby process on 32-bit FreeBSD server. I personally dislike freebsd for a bunch of uncomfortable defaults, sometimes it seems for me that freebsd guys tried to make my life worse. Anyway here I am.

If you have a live ruby process with fancy debug symbols, you should be reading this guide.

What do I do?

Okay,

gdb /usr/local/bin/ruby —core ruby.code

(gdb) bt
#0 0×4809c38c in rb_eval () from /usr/local/lib/libruby18.so.18
#1 0×4809dcac in rb_eval () from /usr/local/lib/libruby18.so.18
#2 0×4809ed8a in rb_eval () from /usr/local/lib/libruby18.so.18
…
#4731 0×48099a02 in ruby_exec () from /usr/local/lib/libruby18.so.18
#4732 0×48099a3e in ruby_run () from /usr/local/lib/libruby18.so.18
#4733 0×0804865f in main ()

Woot!, 4733 stack frames. Okay let’s go to ruby’s source:

static VALUE
rb_eval(self, n)
VALUE self;
NODE *n;

Hmm, NODE* interesting.

typedef struct RNode {
unsigned long flags;
char *nd_file;
…
};

#define FL_USHIFT 11
#define NODE_LSHIFT (FL_USHIFT+8)
#define nd_line(n) ((unsigned int)(((RNODE)→flags>>NODE_LSHIFT)&NODE_LMASK))

I need the line number and the file. As you can see, that information are stored in (RNode*)n->nd_file and (RNode *)n->flags>>19.

Now, I need the second argument of rb_eval function. To get it I’m gonna use info frame

(gdb) info frame
Stack level 1, frame at 0xbfa005a0:
eip = 0×4809dcac in rb_eval; saved eip 0×4809ed8a
called by frame at 0xbfa00910, caller of frame at 0xbfa00230
Arglist at 0xbfa00598, args:
Locals at 0xbfa00598, Previous frame’s sp is 0xbfa005a0
Saved registers:
ebx at 0xbfa0058c, ebp at 0xbfa00598, esi at 0xbfa00590, edi at 0xbfa00594, eip at 0xbfa0059c

It says Stack level 1, frame at 0xbfa005a0 which means 0xbfa005a0 is my first argument, and 0xbfa005a4 is the second. All I need now: printf "%s\n", * ( * ( 0xbfa005a4 ) + 4 ), and:

(gdb) printf “%s\n”, * ( * ( 0xbfa005a4 ) + 4)
/usr/local/lib/ruby/gems/1.8/gems/hpricot-0.6.164/lib/hpricot/parse.rb
(gdb) printf “%d\n”, * ( * ( 0xbfa005a4 ) ) >> 19
235

Nice, isn’t it?

I defined a little macro

define rb_current_file
printf “%s:%d\n”, * ( * ($arg0+4) + 4), * ( * ( $arg0 + 4 ) ) >> 19
end

So I can info frame frame_number_here, and rb_current_file addr to get file and line for this stack frame. I guess there’s a way to write the stack trace dumper, but I don’t want to research it.

Arguments for method call

(gdb) info frame 4573
Stack frame at 0xbfbe9790:
….
(gdb) rb_current_File 0xbfbe9790
myscript.rb:37

Where 37 line is

hpricot = Hpricot(html) unless hpricot

I set it as frame 4573, and went down to #4572 0x480a4ac6 in rb_call () from /usr/local/lib/libruby18.so.18. Okay it’s the actual hpricot call. I need an argument of it. What should I do?. Of course, go to ruby source:

rb_call(klass, recv, mid, argc, argv, scope, self)

As you can see, fifth argument is what I need. I can get it by *(frame+4*4). My frame address is 0xbfbe9400, so I do printf "0x%x\n", *(*((unsigned long*)0xbfbe9400+4)) and see 0xc27c304. It’s a VALUE, so it has type in the first byte: print *0xc27c304 & 0x3f equals 7, which means it’s an RString.

struct RString {
struct RBasic basic;
long len;
char *ptr;
…
};

RBasic is 8 bytes long. So I can get length of the string by print *(0xc27c304+8), and the string by print (char*)*(0xc27c304+12). In my case, all I need to do is dump binary memory file.html 221081600 221301397 and test:

$ irb
irb(main):001:0> require ‘rubygems’
=> true
irb(main):002:0> require ‘hpricot’
=> true
irb(main):003:0> Hpricot(File.read(‘file.html’))
Illegal instruction: 4 (core dumped)

That’s it… Send questions to libc at libc.st

tidy_ffi

2009-02-22T00:00:00+03:00

I started tidy_ffi. It’s a Tidy bindings implemented via FFI gem. FFI is easy and fun.

Problems of current tidy gem.

There’s known bug with tidy API. On December 2006 tidy guys decided to change API, they had added allocator pointer in TidyBuf. So there’re two kinds of tidy libraries in the wild: without allocator (Mac OS X bundled one) and with allocator (newer version that can be found on linux and freebsd).

The second problem it doesn’t work with ruby 1.9. Ruby 1.9 came out with changed dl. FFI should provide ability to use tidy_ffi in rubinius and jruby as well.

Download

To install tidy_ffi type.

gem install tidy_ffi

And that’s it. Of course you need libffi installed, it’s not a gem. Mac OS X users have it installed, others should use distro tools to install it (apt-get, rpm whatever).

Usage

It’s quite easy too.

>> require ‘tidy_ffi’
=> true
>> TidyFFI::Tidy.new(‘test’).clean
=> “<!DOCTYPE html PUBLIC \”-//W3C//DTD HTML 3.2//EN\“>\n<html>\n<head>\n<meta name=\”generator\" content=\n\“HTML Tidy for Mac OS X (vers 31 October 2006 – Apple Inc. build 13), see www.w3.org\”>\n<title></title>\n</head>\n<body>\ntest\n</body>\n</html>\n"

There’s no way to get warnings yet. But several ways to change options for tidy.

More coming soon…

Affleck was the bomb in Phantoms

2009-02-22T00:00:00+03:00

I just watched Phantoms. Awesome movie… There’re some lags, but anyway it’s a pretty awesome movie. If you have time, go ahead and watch it.

It’s kinda test post :)