Creating a http-proxy that can modify the http response before sending it to the client

Question

I'm using wget to grab a something from the web, but I don't want to follow a portion of the page. I thought I could set up a proxy that would remove the parts of the webpage I didn't want to be processed, before returning it to wget but I'm not sure how I would accomplish that.

Is there a proxy that lets me easily modify the http response in python or node.js?

I think more detail is needed. What are you actually trying to do? What do you want to do with the response after modifying it? What do you want to change about the response? Do you want to modify the headers? Or the data? — Acorn
– Acorn, Commented May 6, 2011 at 20:25
@Acorn - I want to change that data in the proxy step of: Internet -> Proxy -> wget — Kit Sunde
– Kit Sunde, Commented May 6, 2011 at 21:09

sintaxi · Accepted Answer · 2011-05-06 20:32:36Z

6

There are several ways you could achieve this goal. This should get you started (using node.js). In the following example I am fetching google.com and replacting all instances of "google" with "foobar".

// package.json file...
{
  "name": "proxy-example",
  "description": "a simple example of modifying response using a proxy",
  "version": "0.0.1",
  "dependencies": {
    "request": "1.9.5"
  }
}

// server.js file...
var http = require("http")
var request = require("request")
var port = process.env.PORT || 8001

http.createServer(function(req, rsp){
  var options = { uri: "http://google.com" }

  request(options, function(err, response, body){
    rsp.writeHead(200)
    rsp.end(body.replace(/google/g, "foobar"))
  })

}).listen(port)

console.log("listening on port " + port)

answered May 6, 2011 at 20:32

sintaxi

1,86814 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Giacomo · Accepted Answer · 2011-05-06 21:40:06Z

0

In nodejs I would fork node-http-proxy and customize the code to my needs.

Much simpler that writing an http proxy from scratch, IMHO.

answered May 6, 2011 at 21:40

Giacomo

11.3k5 gold badges27 silver badges25 bronze badges

Collectives™ on Stack Overflow

Creating a http-proxy that can modify the http response before sending it to the client

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related