Exception in thread "main" java.io.IOException: No input paths specified in job

Question

I am trying to read a json file using spark in Java. The few changes I tried were :

SparkConf conf = new SparkConf().setAppName("Search").setMaster("local[*]");

DataFrame df = sqlContext.read().json("../Users/pshah/Desktop/sample.json/*");

Code:

import java.util.Arrays;
import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;
import org.apache.spark.api.java.function.FlatMapFunction;
import org.apache.spark.api.java.function.Function;
import org.apache.spark.api.java.function.PairFunction;
import org.apache.spark.sql.DataFrame;
import org.apache.spark.sql.SQLContext;
public class ParseData {

public static void main(String args[]){
    SparkConf conf = new SparkConf().setAppName("Search").setMaster("local");
    JavaSparkContext sc= new JavaSparkContext(conf);
    SQLContext sqlContext = new org.apache.spark.sql.SQLContext(sc);
    // Create the DataFrame
    DataFrame df = sqlContext.read().json("/Users/pshah/Desktop/sample.json");
    // Show the content of the DataFrame
    df.show();
}}

Error:

Exception in thread "main" java.io.IOException: No input paths specified in job
at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:198)

I've been unable to replicate your issue. Could You please provide pastebin with the JSON file? — mszymborski
– mszymborski, Commented Jul 23, 2016 at 21:45
Are you sure the path exists? it seems you point to the wrong path — Valijon
– Valijon, Commented Jul 24, 2016 at 0:37
Yes. The path exists. I just tried building the maven project again and it's running fine now. Thank you for the response! — PShah
– PShah, Commented Jul 24, 2016 at 0:50
The problem is the code runs fine for very simple JSON file. But when I am using nested json data . I get the following error : java.lang.RuntimeException: Failed to parse a value for data type StructType() (current token: VALUE_STRING — PShah
– PShah, Commented Jul 24, 2016 at 0:53

Shengbing · Accepted Answer · 2016-12-08 02:28:25Z

0

i wrote the same code, and meet the same problem. i put the people.json file under project directory src/main/resources. the reason is program could not finding the file. after i copy the people.json file to the program's working directory, the program works well

answered Dec 8, 2016 at 2:28

Shengbing

12 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Exception in thread "main" java.io.IOException: No input paths specified in job

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related