4

For the lack of a better example, let's say I want to write a simple client with Rust that could establish a connection and receive data from Twitter's HTTP Streaming API. Is this possible yet? I've been keeping an eye on Iron and Nickel which seem like good frameworks, but I don't think they have this feature yet?

1 Answer 1

6

The http client hyper supports reading responses incrementally (as anything that implements rust's Reader trait), but I wasn't able to find anything to parse the response incrementally, or that implements twitter's particular protocol (to end objecs with \r\n).

That said, I was able to implement a quick'n'dirty proof of concept:

EDIT: See and play with it on github.

use rustc_serialize::json::Json;
use std::str;

pub trait JsonObjectStreamer {
    fn json_objects(&mut self) -> JsonObjects<Self>;
}

impl<T: Buffer> JsonObjectStreamer for T {
    fn json_objects(&mut self) -> JsonObjects<T> {
        JsonObjects { reader: self }
    }
}

pub struct JsonObjects<'a, B> where B: 'a {
    reader: &'a mut B
}

impl<'a, B> Iterator for JsonObjects<'a, B> where B: Buffer + 'a {
    type Item = Json;

    fn next(&mut self) -> Option<Json> {
        let mut line_bytes = match self.reader.read_until(b'\r') {
            Ok(bytes) => bytes,
            Err(_)    => return None,
        };

        if line_bytes.last() == Some(&b'\r') {
            // drop the \r
            line_bytes.pop();

            // skip the \n
            match self.reader.read_char() {
                Ok(_)  => (),
                Err(_) => return None,
            }
        }

        let line = match str::from_utf8(&line_bytes) {
            Ok(line) => line,
            Err(_)   => return None
        };

        Json::from_str(line).ok()
    }
}

Usage: (assuming you have dropped it on a src/json_streamer.rs file on your project)

#![feature(io)]

extern crate hyper;
extern crate "rustc-serialize" as rustc_serialize;

mod json_streamer;

use hyper::Client;

use std::old_io::BufferedReader;
use json_streamer::JsonObjectStreamer;

fn main() {
    let mut client = Client::new();
    let res = client.get("http://localhost:4567/").send().unwrap();

    for obj in BufferedReader::new(res).json_objects() {
        println!("object arrived: {}", obj);
    }
}

I've used this tiny sinatra app to test it:

require 'sinatra'
require 'json'

class Stream
  def each
    hash = { index: 0 }

    loop do
      hash[:index] += 1
      yield hash.to_json + "\r\n"
      sleep 0.5
    end
  end
end

get '/' do
  Stream.new
end
Sign up to request clarification or add additional context in comments.

3 Comments

I can't seem to be able to compile it... getting failed to run custom build command for 'openssl-sys v0.3.5' Process didn't exit successfully: '/home/user/code/hello_rust/target/build/openssl-sys-6694e080744fb2e2/build-script-build' (status=101) --- stderr thread '<main>' panicked at 'Unable to find openssl libraries', /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/openssl-sys-0.3.5/build.rs:46
@Caballero It seems you are missing some openssl dependencies. I suggest you take a look at rust-openssl's readme
Thanks, totally missed the libssl-dev dependency

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.