I've developed a simple application to grab golfer index scores from a website that has no API. The application works but is very slow, with 6 users that require updating takes 60 seconds. I've tried to make my web requests asyncronous to offset some of this lag but it only resulted in a 15% increase in performance.

Description of the code:

On my view I have anchor tag that when clicked hides all the elements in the DOM and loads a preloader, after the preloader is appended to the DOM a AJAX call is executed that calls the UpdatedHandicap method on a controller in my project. From there we await a static method GrabIndexValue. All the code works, it's just very slow.

Possible solution:

This website allows me to input multiple GHIN #s however the result set is in a table with strangely generated xpaths:

//*[@id="ctl00_bodyMP_gvLookupResults_ctl02_lblHI"] which returns index of: 10.5
//*[@id="ctl00_bodyMP_gvLookupResults_ctl03_lblHI"] which returns index of: 9
//*[@id="ctl00_bodyMP_gvLookupResults_ctl04_lblHI"] which returns index of: 13.5

I don't know how to dynamically grab those result sets and parse them properly. So I feel like I'm stuck doing 1 web request per index value.

Async controller method:

public async Task<ActionResult> UpdateHandicap()
        {
            //Fetch all the golfers
            var results = db.Users.ToList();
            //Iterate through the golfers and update their index value based off their GHIN #. We store this
            //value in the database to make our handicap calculation
            foreach (Users user in results)
            {
                user.Index = await Calculations.GrabIndexValue(user.GHID);
                db.Entry(user).State = EntityState.Modified;
                db.SaveChanges();
            }
            return RedirectToAction("Index", "Users");
        }

Method to actually grab the data:

public static async Task<double?> GrabIndexValue(int ghin)
    {
        string url = $"http://xxxxxxxxx/Widgets/HandicapLookupResults.aspx?entry=1&ghinno="+ghin+"&css=default&dynamic=&small=0&mode=&tab=0";
        HtmlWeb web = new HtmlWeb();
        HtmlDocument htmlDoc  = await Task.Run(()=>web.Load(url));

        string index = null;
        try
        { 
            index = htmlDoc.DocumentNode.SelectNodes("//*[@id=\"ctl00_bodyMP_grdClubs\"]/tbody/tr/td[2]")[0].InnerText;
        }
        catch
        { }

        if (index != null)
        {
            var indexResult = Convert.ToDouble(index);
            return indexResult;
        }
        else
        {
            return null;
        }
    }

Any insight into this predicament is greatly appreciated

Thank you