issue parsing json file with Langchain

Question

Need some help.

I have the following json content in a file and would like to use langchain.js and gpt to parse , store and answer question such as

for example:

"find me jobs with 2 year experience" ==> should return a list

"I have knowledge in javascript find me jobs" ==> should return the jobs pbject

I use langchain json loader and I see the file is parse but it say that it find 13 docs . There is only be 3 docs in file . Is the json structure not correct?

Here is snippet of my parse code

const loader = new DirectoryLoader(docPath, {
  ".json": (path) => new JSONLoader(path),
});

const docs = await loader.load();
console.log(docs);
console.log(docs.length);

Here is my input data

[
  {
    "jobid":"job1",
    "title":"software engineer"
    "skills":"java,javascript",
    "description":"this job requires a associate degrees in CS and 2 years experience"
  },
   {
    "jobid":"job2",
    "skills":"math, accounting, spreadsheet",
    "description":"this job requires a degrees in accounting and 2 years experience"
  },
   {
    "jobid":"job3",
    "title":"programmer"
    "skills":"java,javascript,cloud computing",
    "description":"this job requires a ,master degrees in CS and 3 years experience"
  }
  
]

OUTPUT
[
  Document {
    pageContent: 'job1',
    metadata: {
      source: 'langchain-document-loaders-in-node-js/documents/jobs.json',
      line: 1
    }
  },
  Document {
    pageContent: 'software engineer',
    metadata: {
      source: 'langchain-document-loaders-in-node-js/documents/jobs.json',
      line: 2
    }
  },
  Document {
    pageContent: 'java,javascript',
    metadata: {
      source: 'langchain-document-loaders-in-node-js/documents/jobs.json',
      line: 3
    }
  },
  Document {
    pageContent: 'this job requires a associate degrees in CS and 2 years experience',
    metadata: {
      source: 'langchain-document-loaders-in-node-js/documents/jobs.json',
      line: 4
    }
  },
  Document {
    pageContent: 'job2',
    metadata: {
      source: 'langchain-document-loaders-in-node-js/documents/jobs.json',
      line: 5
    }
  },

...

Thank you. Please add your answer so I can give you credit

user2570135
– user2570135

2023-06-19 14:10:28 +00:00
Commented Jun 19, 2023 at 14:10 — user2570135
– user2570135, Commented Jun 19, 2023 at 14:10
Sorry. I did not find a solution.

user2570135
– user2570135

2023-10-30 13:06:41 +00:00
Commented Oct 30, 2023 at 13:06 — user2570135
– user2570135, Commented Oct 30, 2023 at 13:06

O. Jones · Accepted Answer · 2023-06-19 14:30:30Z

1

Your JSON contains a Javascript array of three Javascript objects. Two of them have four properties, and one has three. All the properties have text strings for values. It looks like your parser pulls each property into one of its Documents.

You need to find a way to tell your parser that each Javascript object is one of its Documents.

answered Jun 19, 2023 at 14:30

O. Jones

109k17 gold badges134 silver badges187 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Adrian Mole · Accepted Answer · 2023-11-06 14:11:11Z

1

Having the same issue here despite adding a pointer. I found this GitHub issue, which was opened recently.

An alternative work-around would be to turn JSON into CSV.

edited Nov 6, 2023 at 14:11

Adrian Mole

52.1k193 gold badges61 silver badges101 bronze badges

answered Nov 5, 2023 at 17:10

Antho

312 bronze badges

Collectives™ on Stack Overflow

issue parsing json file with Langchain

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related