1

I am trying to write a code that will generate a values for the table above. These values will be generated while following a set of rules:

  1. The current state and the next state cannot be the same. So if A is the current state then the next state has to be either B or C.

  2. The values will be generated at random every time the program is executed.

  3. All the rows having the same current state, action and next state should have the same rewards.

The problem is that even though I have made lists to choose values from at random, I am still unable to generate values that wont violate rule 1. Every time I run the code, rule 1 is violated. The code for the program can be seen as below.

import random
# Creating lists of variables that will be used to pick a next state at 
random while standing on a current state
NSA = ['B', 'C'] # List of states reachable from A. We use this list to pick 
a next state at random
NSB = ['A', 'C'] # List of states reachable from B. We use this list to pick 
a next state at random
NSC = ['A', 'B'] # List of states reachable from B. We use this list to pick 
a next state at random
Current = ['A', 'B', 'C'] # List of possible current states


RandomList = [-5.0, -4.0, -3.0, -2.0, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0] # List 
of possible rewards for actions taken
Action = ["Clockwise", "Counterclockwise"] # List of possible actions taken 
from a current state

# We need to make sure we know what the current state is, what the action 
will be and what would be the next state and have a reward for it

# Since there are 2 possible states reachable from A after a clockwise 
action, we create and declare 2 variables. These variables will be used to 
pick the next
# state at random. Having 2 variables ensures that all the entries are not 
populated by a single state and thus makins sure we have both states 
populating the column for the action. We then do this for counterclockwise 
action and repeat it for everystate

# Since python doesnt allow to declare variables without instantiation, 
hence we are assigning a value to each state. This wont matter as we will 
later be using the random function to choose a value from the respected 
lists declared above.

# Variables as mentioned above for state A. Note cw = clockwise and ccw = 
counterclockwise
AnScw1 = "B"
AnScw2 = "C"
AnSccw1 = "B"
AnSccw2 = "C"

# Variables as mentioned above for state B. Note cw = clockwise and ccw = 
counterclockwise
BnScw1 = "A"
BnScw2 = "C"
BnSccw1 = "A"
BnSccw2 = "C"

# Variables as mentioned above for state C. Note cw = clockwise and ccw = 
counterclockwise
AnScw1 = "A"
AnScw2 = "B"
AnSccw1 = "A"
AnSccw2 = "B"


# creating variables to pick rewards randomly
a_cw_r1 = random.choice(RandomList)
a_cw_r2 = random.choice(RandomList)
a_ccw_r1 = random.choice(RandomList)
a_ccw_r2 = random.choice(RandomList)
b_cw_r1 = random.choice(RandomList)
b_cw_r2 = random.choice(RandomList)
b_ccw_r1 = random.choice(RandomList)
b_ccw_r2 = random.choice(RandomList)
c_cw_r1 = random.choice(RandomList)
c_cw_r2 = random.choice(RandomList)
c_ccw_r1 = random.choice(RandomList)
c_ccw_r2 = random.choice(RandomList)

current = random.choice(Current) # Pick at randomly a current state from the list Current

if (current=="A") :
action = random.choice(Action) # pick a action at random, from the list of actions
if (action=="Clockwise") : # if clockwise action picked
    AnScw1 = random.choice(NSA) # randomly choose a next state
    AnScw2 = random.choice(NSA) # randomly choose a next state
    # if the two states are the same, then change the values. So if any one of two has B as next state, the other is
    # is assigned C as the next state.
    if (AnScw1 == "B" or AnScw2 == "B") : 
        if (AnScw1 == AnScw2 and AnScw1 == "B") :
            AnScw2 = "C"
            a_cw_b_r1 = random.choice(RandomList) # pick a reward from list at random
            a_cw_b_r2 = random.choice(RandomList) # pick a reward from list at random
    if (AnScw1 == "C" or AnScw2 == "C") :
        if (AnScw1 == AnScw2 and AnScw1 == "C") :
            AnScw2 = "B"
            a_cw_r1 = random.choice(RandomList)
            a_cw_r2 = random.choice(RandomList)

if (action=="Counterclockwise") :
    AnSccw1 = random.choice(NSA)
    AnSccw2 = random.choice(NSA)
    if (AnSccw1 == "B" or AnSccw2 == "B") :
        if (AnSccw1 == AnSccw2 and AnSccw1 == "B") :
            AnSccw2 = "C"
            a_ccw_r1 = random.choice(RandomList)
            a_ccw_r2 = random.choice(RandomList)
    if (AnSccw1 == "C" or AnSccw2 == "C") :
        if (AnSccw1 == AnSccw2 and AnSccw1 == "C") :
            AnSccw2 = "B"
            a_ccw_r1 = random.choice(RandomList)
            a_ccw_r2 = random.choice(RandomList)
if (current=="B") :
action = random.choice(Action)
if (action=="Clockwise") :
    BnScw1 = random.choice(NSB)
    BnScw2 = random.choice(NSB)
    if (BnScw1 == "A" or BnScw2 == "A") :
        if (BnScw1 == BnScw2 and BnScw1 == "A") :
            BnScw2 = "C"
            b_cw_b_r1 = random.choice(RandomList)
            b_cw_b_r2 = random.choice(RandomList)
    if (BnScw1 == "C" or BnScw2 == "C") :
        if (BnScw1 == BnScw2 and BnScw1 == "C") :
            BnScw2 = "B"
            b_cw_r1 = random.choice(RandomList)
            b_cw_r2 = random.choice(RandomList)

if (action=="Counterlockwise") :
    BnSccw1 = random.choice(NSB)
    BnSccw2 = random.choice(NSB)
    if (BnSccw1 == "A" or BnSccw2 == "A") :
        if (BnSccw1 == BnSccw2 and BnSccw1 == "A") :
            BnSccw2 = "C"
            b_ccw_r1 = random.choice(RandomList)
            b_ccw_r2 = random.choice(RandomList)
    if (BnSccw1 == "C" or BnSccw2 == "C") :
        if (BnSccw1 == BnSccw2 and BnSccw1 == "C") :
            BnSccw2 = "B"
            b_ccw_r1 = random.choice(RandomList)
            b_ccw_r2 = random.choice(RandomList)
if (current=="C") :
action = random.choice(Action)
if (action=="Clockwise") :
    CnScw1 = random.choice(NSC)
    CnScw2 = random.choice(NSC)
    if (CnScw1 == "A" or CnScw2 == "A") :
        if (CnScw1 == CnScw2 and CnScw1 == "A") :
            CnScw2 = "B"
            c_cw_r1 = random.choice(RandomList)
            c_cw_r2 = random.choice(RandomList)
    if (CnScw1 == "B" or CnScw2 == "B") :
        if (CnScw1 == CnScw2 and CnScw1 == "B") :
            CnScw2 = "A"
            c_cw_r1 = random.choice(RandomList)
            c_cw_r2 = random.choice(RandomList)

if (action=="Counterclockwise") :
    CnSccw1 = random.choice(NSC)
    CnSccw2 = random.choice(NSC)
    if (CnSccw1 == "A" or CnSccw2 == "A") :
        if (CnSccw1 == CnSccw2 and CnSccw1 == "A") :
            CnSccw2 = "B"
            c_ccw_r1 = random.choice(RandomList)
            c_ccw_r2 = random.choice(RandomList)
    if (CnSccw1 == "B" or CnSccw2 == "B") :
        if (CnSccw1 == CnSccw2 and CnSccw1 == "B") :
            CnSccw2 = "A"
            c_ccw_r1 = random.choice(RandomList)
            c_ccw_r2 = random.choice(RandomList)

As you can see I have created 3 lists NSA, NSB and NSC. I use these 3 lists to pick a value from them at random but I am getting the item that isnt even on the list. So for example look at the following part of the code

NSA = ['B', 'C']
AnScw1 = "B"
AnScw2 = "C"
AnSccw1 = "B"
AnSccw2 = "C"
#calling then later like this
AnScw1 = random.choice(NSA)
AnScw2 = random.choice(NSA)
AnSccw1 = random.choice(NSA)
AnSccw2 = random.choice(NSA)

I should be getting either B or C from AnScw1,AnScw2,AnSccw1,AnSccw2 but instead I also end up with A which is not a item of the list.

The desired output should look like this:

===========================================================================
|| Current State || || Action Taken        || ||  Next State   || Reward ||
===========================================================================
||    A          || ||     Clockwise       || ||      C        || -2.0   || 
||    C          || ||  Counterclockwise   || ||      A        ||  4.0   ||
||    B          || ||  Counterclockwise   || ||      C        ||  4.0   ||
||    A          || ||  Counterclockwise   || ||      C        ||  4.0   ||

Desired Output

4
  • How do you know rule 1 is being violated? I see no statements that will product any output, so when you run this program nothing will happen. Am I missing something? Commented May 1, 2018 at 0:37
  • No matter what I tried, I simply could not format the part of code for the output to this sites requirements. You can see the code for the output in the picture attached showing the desired output. Commented May 1, 2018 at 7:17
  • Your print statements make no sense to me. In each line you hard-code the letter of the current state, but the current state gets randomly selected at each step. So don't you have to print the value of current at each step? Then in your first print statement, for example, the current state is printed as "A" but the next state is the value of the variable BnScw1. BnScw1 is set only if the current state is B, not A. If this was my program I would forget the fancy formatting for the time being and first figure out how to make the logic correct. Commented May 1, 2018 at 12:03
  • the fancy formatting was to give a general idea of the output of the table. the hardcoded values were used just for the output purposes. but thanks a lot for finding out the problem. appearently due to some incorrect installation reasons or whatseoever, my complier did not save any changes I had made and so while the current state was changed, the next state and the reward wasnt. I did a fresh installation and now it works. thanks once again Commented May 1, 2018 at 15:10

1 Answer 1

1

I think to ensure your rule nr 1 is never validated you could use assert statement, something like this:

b = rand() * b

assert a != b 
Sign up to request clarification or add additional context in comments.

1 Comment

the assert statement while works sometimes, throws an exception other times. it doesnt 100% make sure that the rule 1 is never voilated

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.