I am probably doing something very simple, but I cant figure out the trick there.
I have a dataframe, and I want to replace the values in a particular column that exceed a value from zero with some random value. I had thought this was a way of achieving this:
self.dfile['foo'] = np.where(self.dfile['foo'] >= 0, random.uniform(4, 9), self.dfile['foo'])
It seems to be giving the same random value across all values that exceed 0. How do I get different values?
Use numpy.random.uniform with specifying length by length of DataFrame:
v = np.random.uniform(4, 9, size=len(self.dfile))
self.dfile['foo'] = np.where(self.dfile['foo'] >= 0, v,self.dfile['foo'])
Sample:
np.random.seed(123)
dfile = pd.DataFrame({
'foo':[0,5.1,1,0,20.4,10.7],
})
v = np.random.uniform(4, 9, size=len(dfile))
dfile['foo1'] = np.where(dfile['foo'] >= 0, v, dfile['foo'])
dfile['foo2'] = np.where(dfile['foo'].between(0, 10), v ,dfile['foo'])
print (dfile)
foo foo1 foo2
0 0.0 7.482346 7.482346
1 5.1 5.430697 5.430697
2 1.0 5.134257 5.134257
3 0.0 6.756574 6.756574
4 20.4 7.597345 20.400000
5 10.7 6.115532 10.700000
self.dfile['foo'] = np.where(self.dfile['foo'] >= 0 and self.dfile['foo'] <= 10, v, self.dfile['foo'])
& for bitwise AND and also () like self.dfile['foo'] = np.where((self.dfile['foo'] >= 0) & (self.dfile['foo'] <= 10), v, self.dfile['foo'])self.dfile['foo'] = np.where(self.dfile['foo'].between(0, 10), v,self.dfile['foo'])random.uniform(4, 9) returns an integer, which np.where then broadcasts across all rows. Instead, use np.random, which gives an array of specified length:
self.dfile['foo'] = np.where(self.dfile['foo'] >= 0,
np.random.uniform(4, 9, len(self.dfile.index)),
self.dfile['foo'])