Basically you've modified every element in a to reference the list itself. The list is recursively referencing itself:
a[0] # => a
a[0][0] # => a[0], which is a
a[0][0][0] # => a[0][0], which is a[0], which is a
...
(# => is a Rubyism for "this line evaluates to")
Depending on how you look at it it is not infinite. It's more or less just like a piece of paper with the words "please turn over" written on both sides.
The reason that Ruby prints [...] is that it is clever enough to discover that the list is recursive, and avoids going into an infinite loop.
By the way, your usage of each is a bit non-idiomatic. each returns the list, and you usually don't assign this return value to a variable (since you already have a variable referencing it, a in this case). In other words, your code assigns [0,1,2] to a, then loops over a (setting each element to a), then assigns a to a.
a = []; a[0] = a; a[1] = a; a[2] = awill have exactly the same result.blk = proc { |i| Array.new(10, &blk) }; a = Array.new(10, &blk). This causes a SystemStackError though as Ruby can't handle infinite recursion.