How to get key values from HTTP post request using C++ and Boost Asio

Question

I have been given an assignment which involves writing a web server using c++ and the Boost Asio library.

I have got together working server which can send html files back to the client browser using a book called "Boost.Asio C++ Network Programming Cookbook" but I am struggling with handling POST requests from the client.

When a client connects to the server they are given simple HTML form consisting of a username and password field to login in to the server, which is then sent to the server using a POST request.

I have output the contents of the received POST request to the console and I can see all the header information, but I cannot see the form data. I have used Wireshark to inspect the packets and the data is being sent over the network.

The data is being received by the server as a Boost Asio streambuf and I am parsing it to get the requested HTML file by reading it into a vector and then taking the relevant elements such as the method or target.

Does anybody have any suggestions as to where to look for tutorials on how to parse the form data?

The code below is part of the cpp file which parses a POST request and will handle the response based on the contents of the request. the '&request' parameter is the Boost Asio streambuf

I have very little experience in web programming and would be grateful for any advice!

Code to parse requests

// Prepare and return the response message.
// Parse the request from the client to find requested document 
std::istream buffer(&request);
std::vector<std::string> parsed((std::istream_iterator<std::string>(buffer)), std::istream_iterator<std::string>() );

Handling POST requests


else if (parsed.size() >= 3 && parsed[0] == "POST") {

            htmlFile = "/files.html";

            // Retrieve files from server file system. The second element in 'parsed' vector is file name
            std::ifstream fileStream(".\\directory" + htmlFile);

            // If the file exists then iterate it and assign the value to the content string variable, else return 404.
            if (fileStream.good()) {
                std::string fileContents((std::istreambuf_iterator<char>(fileStream)), std::istreambuf_iterator<char>());
                content = fileContents;
                code = "200 ok";
            }
            else {
                std::ifstream fileStream(".\\directory\\404.html");
                std::string fileContents((std::istreambuf_iterator<char>(fileStream)), std::istreambuf_iterator<char>());
                content = fileContents;
                code = "404";
            }// End of nested if-else statement 

        }// End of else-if statement
        else {
            std::ifstream fileStream(".\\directory\\401.html");
            std::string fileContents((std::istreambuf_iterator<char>(fileStream)), std::istreambuf_iterator<char>());
            content = fileContents;
            code = "401";
            // Write bad request to log file for security audits if not "GET" request
            logging.logAction("Illegal request by client IP " + m_sock->remote_endpoint().address().to_string());

        }//End of if-else statement

        std::ostringstream oss;
        oss << "GET HTTP/1.1 " << code << " \r\n";
        oss << "Cache-Control: no-cache, private" << "\r\n";
        oss << "Content-Type: text/html" << "\r\n";
        oss << "Content-Length: " << content.size() << "\r\n";
        oss << "\r\n\r\n";
        oss << content;

        response = oss.str().c_str();

Are you sure you read the post data? You did not post this code. Beside of this have a look at boost::beast it cointains an example webserver. — mkaes
– mkaes, Commented Jan 9, 2020 at 8:42
"I have used Wireshark to inspect the packets and the data is being sent over the network." - don't use Wireshark. Use Fiddler to debug web traffic. Wireshark is for raw TCP. Fiddler provides better diagnostics for HTTP traffic. — selbie
– selbie, Commented Jan 9, 2020 at 8:47
The code that output the post data was just a for loop to iterate the vector called parsed. The code posted is missing the logic to check the form data at the moment as I haven't managed to write it — Nick Baker
– Nick Baker, Commented Jan 9, 2020 at 8:47
@selbie Thank you, I will try that and see what I can find out! — Nick Baker
– Nick Baker, Commented Jan 9, 2020 at 8:48
Remember, TCP is a stream protocol, not a message protocol. Just because the client did a single "send" for 1000 bytes doesn't mean your corresponding "Recv" on the server side will receive all 1000 bytes at once. It's the most common misunderstanding about sockets. — selbie
– selbie, Commented Jan 9, 2020 at 8:57

sehe · Accepted Answer · 2020-01-09 10:00:33Z

HTTP is a linewise protocol. Samples: https://www.tutorialspoint.com/http/http_requests.htm

POST /cgi-bin/process.cgi HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Host: www.tutorialspoint.com
Content-Type: application/x-www-form-urlencoded
Content-Length: length
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive

licenseID=string&content=string&/paramsXML=string

You need to be more specific with the parsing than putting each whitespace separated "word" into a vector.

Start with something like this:

Live On Coliru

#include <iostream>
#include <iomanip>
#include <boost/asio.hpp>

int main() {
    boost::asio::streambuf request;
    {
        std::ostream sample(&request);
        sample <<
            "POST /cgi-bin/process.cgi HTTP/1.1\r\n"
            "User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)\r\n"
            "Host: www.tutorialspoint.com\r\n"
            "Content-Type: application/x-www-form-urlencoded\r\n"
            "Content-Length: 49\r\n"
            "Accept-Language: en-us\r\n"
            "Accept-Encoding: gzip, deflate\r\n"
            "Connection: Keep-Alive\r\n"
            "\r\n"
            "licenseID=string&content=string&/paramsXML=string"
            ;
    }

    std::istream buffer(&request);
    std::string line;

    // parsing the headers
    while (getline(buffer, line, '\n')) {
        if (line.empty() || line == "\r") {
            break; // end of headers reached
        }
        if (line.back() == '\r') {
            line.resize(line.size()-1);
        }
        // simply ignoring headers for now
        std::cout << "Ignore header: " << std::quoted(line) << "\n";
    }

    std::string const body(std::istreambuf_iterator<char>{buffer}, {});

    std::cout << "Parsed content: " << std::quoted(body) << "\n";
}

Printing

Ignore header: "POST /cgi-bin/process.cgi HTTP/1.1"
Ignore header: "User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)"
Ignore header: "Host: www.tutorialspoint.com"
Ignore header: "Content-Type: application/x-www-form-urlencoded"
Ignore header: "Content-Length: 49"
Ignore header: "Accept-Language: en-us"
Ignore header: "Accept-Encoding: gzip, deflate"
Ignore header: "Connection: Keep-Alive"
Parsed content: "licenseID=string&content=string&/paramsXML=string"

In addition to Sehe's response: HTTP uses the Json format, so you can use a Json library, there are a lot

Pedro Vicente · Accepted Answer · 2020-09-16 06:52:31Z

simple ASIO HTTP parser that reads "Content-Length: " and then proceeds to use that to read the rest of the non header part (the POST message)

int http::parse(asio::ip::tcp::socket& sock, http_msg_t& http)
{
  std::string line;
  asio::streambuf buf;
  std::stringstream ss;
  asio::error_code error;
  size_t content_size = 0;
  size_t read_left = 0;
  try
  {
    //read until end of HTTP header
    //Note:: after a successful read_until operation, the streambuf may contain additional data 
    //beyond the delimiter. An application will typically leave that data in the streambuf for a subsequent 
    //read_until operation to examine.

    asio::read_until(sock, buf, "\r\n\r\n");
    std::istream stream(&buf);
    while (std::getline(stream, line) && line != "\r")
    {
      http.header.push_back(line);
    }

    //store method and url
    line = http.header.at(0);
    http.method = http::http_get_method(line);
    http.url = http::http_get_url(line);

    //find 'Content-Length'
    for (int idx = 0; idx < http.header.size(); idx++)
    {
      line = http.header.at(idx);
      if (line.find("Content-Length: ") != std::string::npos)
      {
        size_t start = line.find(":");
        start += 2; //space
        size_t end = line.find("\r");
        std::string s = line.substr(start, end - 1);
        try
        {
          content_size = std::atoi(s.c_str());
        }
        catch (std::invalid_argument&)
        {
          events::log("invalid Content-Length");
          return -1;
        }
        http.content_size = content_size;
      }
    }

    if (http.content_size == 0)
    {
      //nothing to read; not a POST; must be an acknowledgement response 
      return 0;
    }

    //read end of message left
    //dump whatever content we already have
    if (buf.size() > 0)
    {
      ss << &buf;
      std::string s = ss.str();
      read_left = content_size - s.size();
    }
    else
    {
      read_left = content_size;
    }

    //asio::read reads exact number of bytes
    size_t recv = asio::read(sock, buf, asio::transfer_exactly(read_left));
    ss << &buf;
  }
  catch (std::exception& e)
  {
    events::log(e.what());
    return -1;
  }
  http.msg = ss.str();
  return 0;
}

How to call this function? What software tool(s) should I call to execute this?

Collectives™ on Stack Overflow

How to get key values from HTTP post request using C++ and Boost Asio

2 Answers 2

1 Comment

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related