0
$\begingroup$
I have written a code for particle swarm optimization to tune the parameters of classification model. However, on every run I get different values of C and gamma.

Below is the code:

    ##Import Data
df = pd.read_excel('C:/Users/Ram Prakash/Downloads/Data.xlsx', sheet_name = 'Binary')
from sklearn.preprocessing import LabelEncoder
le = LabelEncoder()
X = df.drop('FaultType', axis = 1)
y = df['FaultType']
y = le.fit_transform(y)

from imblearn.over_sampling import SMOTE
smote = SMOTE()
X_sm, y_sm = smote.fit_resample(X, y)
#print(X_sm, y_sm)


# Train Test split
X_train, X_test, y_train, y_test = train_test_split(X_sm, y_sm, stratify = y_sm, 
test_size=0.2, random_state = None)
print(X_train.shape, X_test.shape)
scaler = StandardScaler().fit(X_train)
X_train, X_test = scaler.transform(X_train), scaler.transform(X_test)


##Classification by Default parameters
# Fit SVM classifier
clf_default = SVC(kernel='rbf')
clf_default.fit(X_train, y_train)
print('(Cross Validation) AUC Score:', np.mean(cross_val_score(estimator=clf_default, 
X=X_train, y=y_train, cv=5, scoring='roc_auc')))

# Show result
print('(Training set) Confusion Matrix:')
print(confusion_matrix(y_train, clf_default.predict(X_train)))
print('Training set accuracy', accuracy_score(y_train, clf_default.predict(X_train)))
print('(Training set) AUC Score:', roc_auc_score(y_train, clf_default.predict(X_train)))

# Show result
print('(Test set) Confusion Matrix:')
print(confusion_matrix(y_test, clf_default.predict(X_test)))
print(accuracy_score(y_test, clf_default.predict(X_test)))
print('(Test set) AUC Score:', roc_auc_score(y_test, clf_default.predict(X_test)))

##Classification of Parameters by PSO
# Set parameters
C_range = [100, 10000]
gamma_range = [0.0001, 0.1]
para = {'C': C_range, 'gamma': gamma_range}

para_pso = PSOSearchCV_SVM(estimator=SVC(kernel='rbf'), param_grid=para, 
scoring='roc_auc',cv=5, n_jobs=3, verbose=False, pso_size= 30, pso_max_iter= 30, 
velocity_scale=.5)
para_pso.fit(X_train, y_train)
print('Best parameters:', para_pso.best_param)

# Fit SVM classifier
clf_pso = SVC(kernel='rbf', C=para_pso.best_param['C'], 
gamma=para_pso.best_param['gamma'])
clf_pso.fit(X_train, y_train)
print('(Cross Validation) AUC Score:', np.mean(cross_val_score(estimator=clf_pso, 
 X=X_train, y=y_train, cv=5, scoring='roc_auc')))


# Show result
print('(Test set) Confusion Matrix:')
print(confusion_matrix(y_test, clf_pso.predict(X_test)))
print(accuracy_score(y_test, clf_pso.predict(X_test)))

Please suggest how can I get around this problem and why am I getting different values for C and gamma on every run.

$\endgroup$
6
  • $\begingroup$ What is PSOSearchCV_SVM? $\endgroup$ Commented May 4, 2023 at 19:18
  • $\begingroup$ You haven't set any random_states, so there will be natural random variation; are you observing more dramatic changes than you expect? $\endgroup$ Commented May 4, 2023 at 20:12
  • $\begingroup$ Hi @BenReiniger PSOSearchCV_SVM is a class created defined all the parameters and the PSO algorithm. Also, even when I set the random state the values are still changing, which again I cannot understand as to why this might be happening. Yes, the observations are dramatic, I get the same accuracy or fitness function value but different C and gamma values. $\endgroup$ Commented May 4, 2023 at 20:40
  • $\begingroup$ The SMOTE also has random states. $\endgroup$ Commented May 5, 2023 at 2:02
  • $\begingroup$ There are a lot of modules with random states in your code, so as general guidance I suggest checkpoining your data between each module and compare at which point they are not identical among runs. $\endgroup$ Commented May 5, 2023 at 9:38

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.