pd.Grouper creates full-length bins, even when there’s no data in the tail bin (like 16:00–16:30). This is normal behavior for time-based grouping.
Try using the label and closed parameters.
filtered = self.df.groupby(
pd.Grouper(key='Datetime', freq='30min', origin='start', label='right', closed='left')
).mean().reset_index()
or simply drop the nulls when you aggragte/get the mean
self.other_df['Avg_volatility'] = filtered['volatility'].mean().dropna()