ValueError string to float when retrieving float32 from Netcdf file using Netcdf4 in python

Question

I am using netcdf4 in python 2.7 on a windows7 machine. I have loaded numpy recarrays into a netcdf file I created and have subsequently retrieved the data several times. Then, for some unknown reason when I try to retrieve the data I get a ValueError could not convert string to float:

The code that is being used to retrieve the data is:

def getNetCDFGroupVarData(NCfilename, GroupPath, Variable):
"""  ==============================================================
TITLE:      getNetCDFGroupVarData    

DESCR:      for a valid variable on the specified path in a NetCDF file
            returns a data vector 

ARGS:       NCfilename : netcdf4 file path and name
            GroupPath : group path 
            Variable : variable name 

RETURN:     VarData: vector of variable data

DEPEND:     netCDF4.Dataset     

=======================================================================  
"""
# get rootgroup and group from which to return attributes
if os.path.isfile(NCfilename):
    RG = Dataset(NCfilename, 'a')
    G = giveListEndGroup(RG,GroupPath)

    # retrieve variable data from group
    keyVar = G.variables.keys()
    print(keyVar)
    kvlen = len(keyVar)
    var = unicode(Variable) 
    if kvlen > 0 :
        print('variable name: ',var)
        V =  G.variables[var]
        print V.dtype
        print V.shape
        print V.dimensions
        VarData = V[:]          #====== Error raised here ==============
    else:
        print('no keys found')
        VarData = None

    RG.close()
    return VarData

The print outputs and error stack I get when calling this function are:

[u'time', u'SECONDS', u'NANOSECONDS', u'Rg', u'Ts1', u'Ts2', u'Ts3', u'V_log', u'T_log']
('variable name: ', u'time')
float64
(88872,)
(u'time',)
variable:  time  does not exist
Unexpected error: <type 'exceptions.ValueError'>
Traceback (most recent call last):
  File "C:\Users\rclement\Documents\My Dropbox\Code\python\NCTSutil\Panel_NCTS_structure.py", line 69, in tree_path_changed
    pub.sendMessage('NetcdfTS.group.specified', arg1=pathlist )
  File "C:\Python27\lib\site-packages\pubsub\core\kwargs\publisher.py", line 27, in sendMessage
    topicObj.publish(**kwargs)
  File "C:\Python27\lib\site-packages\pubsub\core\kwargs\publishermixin.py", line 24, in publish
    self._publish(msgKwargs)
  File "C:\Python27\lib\site-packages\pubsub\core\topicobj.py", line 376, in _publish
    self.__sendMessage(data, self, iterState)
  File "C:\Python27\lib\site-packages\pubsub\core\topicobj.py", line 397, in __sendMessage
    self._mix_callListener(listener, data, iterState)
  File "C:\Python27\lib\site-packages\pubsub\core\kwargs\publishermixin.py", line 64, in _mix_callListener
    listener(iterState.filteredArgs, self, msgKwargs)
  File "C:\Python27\lib\site-packages\pubsub\core\kwargs\listenerimpl.py", line 43, in __call__
    cb(**kwargs)
  File "C:\Users\rclement\Documents\My Dropbox\Code\python\NCTSutil\NetcdfTimeSeries.py", line 70, in listner_group
    atime = self.GetSelectedVariableData(pathlist, u'time')
  File "C:\Users\rclement\Documents\My Dropbox\Code\python\NCTSutil\NetcdfTimeSeries.py", line 307, in GetSelectedVariableData
    VarData = MNU.getNetCDFGroupVarData(self.filename, GroupPathList, variable )
  File "C:\Users\rclement\Documents\My Dropbox\Code\python\NCTSutil\MyNetcdfUtil.py", line 304, in getNetCDFGroupVarData
    VarData = V[:]  
  File "netCDF4.pyx", line 2949, in netCDF4.Variable.__getitem__ (netCDF4.c:36472)
  File "netCDF4.pyx", line 2969, in netCDF4.Variable._toma (netCDF4.c:36814)
ValueError: could not convert string to float:

When I use other netcdf utilities (i.e. panolpy) I can access the data. Does anyone have a clue why netcdf4 would be throwing this excpetion - or worse - how it could have inserted a string in my float32 field in the netcdf file?

I suggest you catch the ValueError exception and examine the problem string — ali_m
– ali_m, Commented Mar 5, 2015 at 14:06

RJCL · Accepted Answer · 2015-03-05 14:41:55Z

2

From the traceback the problem was occurring in the "_toma" Netcdf4 function which converts the data to a masked array. When reading the file with other utilities (eg. NCDUMP) I had no problem accessing the data.

At the moment I believe the problem occurred because I had an unassigned 'missing_value' attribute for the variable. Apparently, if there is no 'missing_value' attribute Netcdf4 defaults to a missing value appropriate for the dtype. In my implementation the 'missing_value' attribute was being exposed for editing via a wxpyhton grid control. When the edited attributes in the grid were written back to the netcdf file the empty grid cell was returning either a None object or wx.emptyString, which Netcdf4 attempted to insert into the float type in the netcdf file.

answered Mar 5, 2015 at 14:41

RJCL

3573 silver badges16 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

ValueError string to float when retrieving float32 from Netcdf file using Netcdf4 in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related