After trying to switch back two UART pins to GPIO by using the Serial.end() function, the pins do not respond to digitalWrite commands.
I have found out that by using gpio_matrix_out(gpio_number, SIG_GPIO_OUT_IDX, false, false); I am able to use the pin again after ending the Serial.