Extract array values into a CSV file

Question

Hi I have the following code from the site: https://earthscience.stackexchange.com/questions/23947/world-elevation-data-as-csv

import numpy as np
from netCDF4 import Dataset
import matplotlib.pyplot as plt
import pandas as pd
from mpl_toolkits.basemap import Basemap
from matplotlib.patches import Path, PathPatch

csv_data = np.loadtxt('csv_data(lat/long/value).csv',skiprows=1,delimiter=',')
num_el = csv_data[:,0]
lat = csv_data[:,1]
lon = csv_data[:,2]
value = csv_data[:,3]

data = Dataset("elevation_data.grd",'r')
lon_range = data.variables['x_range'][:]
lat_range = data.variables['y_range'][:]
topo_range = data.variables['z_range'][:]
spacing = data.variables['spacing'][:]
dimension = data.variables['dimension'][:]
z = data.variables['z'][:]
lon_num =  dimension[0]
lat_num =  dimension[1]

etopo_lon = np.linspace(lon_range[0],lon_range[1],dimension[0])
etopo_lat = np.linspace(lat_range[0],lat_range[1],dimension[1])
topo = np.reshape(z, (lat_num, lon_num))

height = np.empty_like(num_el)
for i in range(len(num_el)): 
    desired_lat_idx = np.abs(etopo_lat - lat[i]).argmin()
    desired_lon_idx = np.abs(etopo_lon - lon[i]).argmin()
    height[i] = topo[desired_lat_idx,desired_lon_idx]
height[height<0]=0 

dfl= pd.DataFrame({
    'Latitude' : desired_lat_idx.reshape(-1),
    'Longitude': desired_lon_idx.reshape(-1),
    'Altitude': height.reshape(-1)
});
dfl.to_csv(path)

With this I can extract the elevation data that are in a file (are attached to the lat/long data from another CSV file. Now I want to write a new CSV file which is structured as follows Long/lat/Value/Altitude. I have tried it with the snipped

dfl= pd.DataFrame({
    'Latitude' : desired_lat_idx.reshape(-1),
    'Longitude': desired_lon_idx.reshape(-1),
    'Altitude': height.reshape(-1)
});
dfl.to_csv(path)

But I get the error

>>> dfl= pd.DataFrame({
...     'Latitude' : desired_lat_idx.reshape(-1),
...     'Longitude': desired_lon_idx.reshape(-1),
...     'Altitude': height.reshape(-1)
... });
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\UsersName\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\frame.py", line 637, in __init__
    mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
  File "C:\UsersName\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\internals\construction.py", line 502, in dict_to_mgr
    return arrays_to_mgr(arrays, columns, index, dtype=dtype, typ=typ, consolidate=copy)
  File "C:\UsersNameAppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\internals\construction.py", line 120, in arrays_to_mgr
    index = _extract_index(arrays)
  File "C:\UsersName\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\core\internals\construction.py", line 674, in _extract_index
    raise ValueError("All arrays must be of the same length")
ValueError: All arrays must be of the same length
>>>

how can I get around this?

Rabinzel · Accepted Answer · 2022-06-23 13:47:24Z

1

desired_lat_idx and desired_lon_idx will be overwritten each loop, so you'll have one value for each of them two (value from the last loop of len(num_el) while your height will have the same length as num_el since you change the value at index i in every loop. Try to print len of desired_lat_idx, desired_lon_idx and height to check if I'm right.

Try:

height = np.empty_like(num_el)
for i in range(len(num_el)): 
    lat_idx = np.abs(etopo_lat - lat[i]).argmin()
    lon_idx = np.abs(etopo_lon - lon[i]).argmin()
    height[i] = topo[lat_idx,lon_idx]
height[height<0]=0 

dfl= pd.DataFrame({
    'Latitude' : lat.reshape(-1),
    'Longitude': lon.reshape(-1),
    'Altitude': height.reshape(-1)
})

edited Jun 23, 2022 at 13:47

answered Jun 23, 2022 at 11:56

Rabinzel

7,9533 gold badges12 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

11 Comments

Weiss Over a year ago

Do you mean that:

Weiss Over a year ago

>>> print(len(desired_lat_idx)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: object of type 'numpy.int64' has no len() >>> print(len(desired_lon_idx)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: object of type 'numpy.int64' has no len() >>> print(len(height)) 360000

Rabinzel Over a year ago

oops, it proves my point but I didn't think of that. Yes, desired_lat_idx is one integer which obviously has no len. As you can see you want to create a df which has two columns of length 1 and one column of length 360000

Weiss Over a year ago

did your code snipded solve my problem? Yes I can see your point however I#m new at coding and dont know how to extract the values right!

Weiss Over a year ago

I tryed youre snipped but without sucsess

|

Collectives™ on Stack Overflow

Extract array values into a CSV file

1 Answer 1

11 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

11 Comments

Your Answer

Sign up or log in

Post as a guest

Related