Calculate average of an array with objects that has a nested Object

The problem with your current code lies in how you build the reformattedArray variable. First, notice that your map function implicitly returns undefined whenever that year is missing from the current object:

let reformattedArray = mathid1.map(obj => {
  if (obj["values"][year]) {
    return obj["values"][year]
  }

  // There is an implicit return undefined, right here...
})

When you use the array .map method, every item of the array will be replaced by the return value of the map function. In the case that the year is not present, it will not go into the if block, and so it implicitly returns undefined upon reaching the end of the function.

So, ultimately all you have to do is remove the undefined entries from this array, and your code will work as-is.

One way to do that is to just use .filter(Boolean) on the array, which removes any falsey entries (which undefined is). Eg:

let reformattedArray = mathid1.map(obj => {
  /* code here */
}).filter(Boolean); // Note the filter here...

Here is your snippet with that modification:

const testObject = [{
    id: 4,
    course: "math",
    values: {
      2017: 8,
      2018: 9
    }
  },
  {
    id: 5,
    course: "English",
    values: {
      2017: 8,
      2018: 9
    }
  },
  {
    id: 4,
    course: "math",
    values: {
      2017: 5,
      2019: 7
    }
  },
  {
    id: 4,
    course: "english",
    values: {
      2017: 5,
      2019: 7
    }
  },
]
//First I filter out the id 4 and course Math
const mathid1 = testObject.filter((e) => e.id === 4 && e.course === "math");
//I than find all the different years
const ArrayOfAllYears = []
mathid1.map((element) => {
  ArrayOfAllYears.push(Object.keys(element.values));
})
//I here find all the different years
const withDuplicates = ArrayOfAllYears.reduce(function(arrayOne, arrayTwo) {
  return arrayOne.concat(arrayTwo);
}, []);

const withoutDuplicates = Array.from(new Set(withDuplicates));
//Here I just create the calculate average function
const Result = {}
const calculateAverage = (array) => {
  const sum = array.reduce((a, b) => a + b);
  return sum / array.length;
};
const newObj = {}
withoutDuplicates.map((year) => {
  let reformattedArray = mathid1.map(obj => {
    if (obj["values"][year]) {
      return obj["values"][year]
    }
  }).filter(Boolean)
  newObj[year] = calculateAverage(reformattedArray)
})
console.log(newObj)
// I want to calculate the average of the mathid1 values and return it on a Object like {2017:..,2018..}

1. Group items by year.
2. Calculate average.

const items=[{
  course: "math",
  id: 4,
  values: {
    2017: 8,
    2018: 9
  }
}, {
  course: "math",
  id: 4,
  values: {
    2017: 5,
    2019: 7
  }
}]

const groupedValues=items.reduce((groupedValues,item)=>{
  Object.entries(item.values).forEach(([year,value])=>{
      if(groupedValues[year]){
        groupedValues[year]={value:groupedValues[year].value+value,items:groupedValues[year].items+1};
      } else {
        groupedValues[year]={value,items:1};
      }
  });
  return groupedValues;
},{})

console.log(groupedValues);

const result = Object.entries(groupedValues).reduce((result,item)=>{
  result[item[0]]=item[1].value/item[1].items;
  return result;
},{})

console.log(result);

I would recommend extracting the years information into a map:

/** @type {Map<string, number[]} */
const years = new Map();

testObject.forEach((obj) => {
  Object.keys(obj.values).forEach((key) => {
    if (!years.has(key)) years.set(key, []);
    years.set(key, [...years.get(key), obj.values[key]]);
  });
});

Then you can simply loop over the map and create the resulting object:

const result = {};
years.forEach((values, key) => {
  Object.defineProperty(result, key, {
    value: values.reduce((acc, val) => acc + val) / values.length,
    enumerable: true,
  });
});

console.log(result);

It should output:

{ '2017': 6.5, '2018': 9, '2019': 7 }