26

I've made a bash script which I run every hour with crontab, and I need to store one variable so that I can access it the next time I run it. The script changes the variable every time it runs, so I can't hardcode it in. Right now I am writing it to a txt file and then reading it back. Is there a better way to do it than this? And the way I am reading the txt file is something I found on here, I don't understand it, and it's kinda clunky. Is there not a built in command for this? Anyway, here's the applicable code, with some of the variables changed to make it easier to read. 

while read x; do
  var=$x
done < var.txt

# Do some stuff, change var to a new value

echo $var > var.txt

The variable is only a single integer, so the text file feels overkill.

Improve this question
1
  • 4
    The short text file holding persistent data is exactly the way to do it, nothing clunky about it. Commented Sep 9, 2012 at 9:30

7 Answers 7

Reset to default
21

There's no need to use var; x will be in scope for the current shell. Alternately, 

read var < var.txt
# do stuff with var
echo $var > var.txt

I recommend using a simple text file to store the variable. However, there is the (highly questionable) option of a self-modifying script. FOR ENTERTAINMENT PURPOSES ONLY!

#!/bin/bash

read val < <( tail -n 1 "$0" )

(( val++ ))
echo "$val"

tmp=$(mktemp /tmp/XXXXXXX)
sed '$s/.*/'$val'/' "$0" > "$tmp"
mv "$tmp" "$0"

exit
0

The key is to have the next-to-last line be the exit command, so nothing after it will execute. The last line is the variable value you want to persist. When the script runs, it reads from its own last line. Before it exits, it uses sed to write a copy of itself toa temp file, with the last line modified with the current value of the persistent value. Then we overwrite the current script with the temp file (assuming we will have permission to do so).

But seriously? Don't do this.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

Thanks, that's a lot simpler. Are text files the best way to do this?
@shardbearer: It depends. You can chose from file, database, other more exotic java/ruby/python object-persistence stores, or keep your program running in an infinite loop (with appropriate sleep times embedded) and store the value in memory, depending on what your overall requirements are. File persistence is usually good enough. If you're getting into a mongo system with thousands of files and dirs (as have been recently posited here on S.O.), then you'll need to spend some time finding the right technology. Good luck to all.
Instead of modifying the last line, you could just append the new values with echo "$val" >> "$0". It's even more terrible (but makes the code much simpler) :) You could also use ed to edit the source file, instead of the horrible combo tmp/sed/mv.
Hahah i really like this response
An explanation of why this shouldn't be done is necessary.
|
6

I know this is an old question. But, I still decide to post my solution here in the hope that it might be helpful to others who come here in search of a way to serialize env vars between sessions.

The simple way is just write "var_name=var_value" into a file, say "./environ". And then "source ./envrion" in following sessions. For example:

echo "var1=$var1" > ./environ

A more comprehensive (and elegant?) way which persist all attributes of variables is to make use of "declare -p":

declare -p var1 var2 > ./environ
# NOTE: no '$' before var1, var2

Later on, after "source ./envrion" you can get var1 var2 with all attributes restored in addition to its value. This means it can handle arrays, integers etc.

One caveat for the "declare -p xx", though: if you wrap the "source ./environ" into a function, then all sourced variables are visible within the function only because "declare" by default declares variables as local ones. To circumvent this, you may either "source" out of any function (or in your "main" function) or modify the ./environ to add "-g" after declare (which makes corresponding variable global). For instance:

sed -i 's/^declare\( -g\)*/declare -g/' ./environ
# "\( -g\)?" ensure no duplication of "-g"
Improve this answer

2 Comments

This is a simple approach, and I use it myself. It does demand care, if untrusted users are able to write to the file (e.g. if it's in /tmp and not initially created), because it contains the full capabilities of the sourcing shell. Take care!
declare -p is much better than echo "var1=$var1" > ./environ, it escapes everything correctly and adds -x to declaration if variable is exported.
4

Depending on your use case this might be overkill, but if you need to store and keep track of multiple variables (or from multiple scripts) then consider using sqlite which has a command line interface (sqlite3), and which is usually preinstalled ootb on linux/macos systems.

DB='storage.db'

KEY1='eurusd'
VAL1=1.19011

KEY2='gbpeur'
VAL2=1.16829

# create table if not present (ONLY NEEDS TO BE RUN ONCE)
QUERY_CREATE="CREATE TABLE IF NOT EXISTS records (id INTEGER PRIMARY KEY, name TEXT NOT NULL, value NUMERIC NOT NULL);"
sqlite3 "$DB" "$QUERY_CREATE"

# write a key-value pair to database (creates a new row each time)
QUERY_INSERT="INSERT INTO records(name, value) VALUES ('${KEY1}', '${VAL1}');"
sqlite3 "$DB" "$QUERY_INSERT"

# write a key-value pair to database (REPLACE previous value!)
# using 42 as a hard-coded row ID
QUERY_REPLACE="REPLACE INTO records(id, name, value) VALUES (42, '${KEY2}', '${VAL2}');"
sqlite3 "$DB" "$QUERY_REPLACE"

# read value from database
QUERY_SELECT1="SELECT value FROM records WHERE name='${KEY1}';"
QUERY_SELECT2="SELECT value FROM records WHERE name='${KEY2}';"

echo "***** $KEY1 *****"
# store db value in a variable
db_value1=$(sqlite3 "$DB" "$QUERY_SELECT1")
echo $db_value1
## OUTPUT: 1.19011

echo "***** $KEY2 *****"
db_value2=$(sqlite3 "$DB" "$QUERY_SELECT2")
echo $db_value2
## OUTPUT: 1.16829

NOTE: If you do not explicitly pass the row ID then a new row will be added on each script invocation. To always update into the same row use REPLACE INTO with an explicit ID (e.g. 42 as can be seen in the REPLACE INTO... statement). Run the script multiple times to see how the output differs for KEY1 and KEY2.

NOTE2: In this example the values are numeric, if you need to store strings then in CREATE TABLE instead of NUMERIC use TEXT.

And if you want an open-source GUI for visualising the database then DB Browser for SQLite is available for mac/linux/windows (there are dozens more).

Importing/Exporting data

# Export values to an SQL dump
sqlite3 storage.db .dump > data.sql

# Import values from an SQL dump
sqlite3 storage.db < data.sql

# Export to CSV file with custom query
sqlite3 storage.db ".output data.csv" ".mode csv"  "SELECT * FROM mytable42;" ".exit"

# Import from CSV file
sqlite3 storage.db  ".mode csv"  ".import data.csv mytable42" ".exit"

You can use over a dozen output formats including JSON, CSV, HTML, TSV, markdown, etc, for details see:

Improve this answer

Comments

3

1- You can simplify your script, as you only have one variable 

var=`cat var.txt`
# Do some stuff, change var to a new value   
echo $var > var.txt

2- You can store your variable in the environment:

export var

# Do some stuff, change var to a new value

But you'll need to prompt it . script.ksh (dot at the beggining). But it shouldn't have 'exit' in it and i'm not sure this would work in cron...

Improve this answer

Comments

0

To store multiple variables between runs, a solution I considered is to save them under the format my_var=my_value in a separated file.

Then, I include two function to set and retrieve the variables

  1. In the file storing the variables and their values:

Let's call this file context.dat

# Here I store the variables and their values
my_var_x=1
my_var_y=boo
my_var_z=0
  1. In the actual script:

Let's call the file multiple_run.sh

context=./context.dat

function update_variables(){
    # update the variable context
    source $context
}

function set_variable(){
    # store variable
    variable=$1 #variable to be set
    value=$2    # value to give to the value
    # modify the file storing the value
    sed -i 's/'${variable}'.*/'${variable}'='${value}'/' $context
}

##################
# Test code
echo var_x
update_variables
echo var_x
# do something
set_variable var_x 2
echo $var_x

This is one approach among other. With such method, you need to create the storing file before and create each line for each variable. Besides, the context.dat is a priori accessible by any other script.

Improve this answer

Comments

0

Just discovered this great simple project (a rewritten fork). A simple yet powerful key/value pair store for bash. Looks perfect. Behind the scenes each database is a directory, each key is a file, and the values are in the file.

https://github.com/imyller/kv-sh

  • Tiny key-value database
  • Configurable database directory (default: ~/.kv-sh)
  • Used by importing functions via $ . ./kv-sh
  • Full database dump/restore
  • Support for secondary read-only defaults database
    . ./kv-sh                  # import kv-sh functions (use default database directory; see
                                 configuration environment variables for available options)
    kvset <key> <value>        # assign value to key
    kvget <key>                # get value of key
    kvdel <key>                # delete key
    kvexists <key>             # check if key exists
    kvkeys {-l|-d|-a}          # list all keys (-l local only, -d default only, -a all (default))
    kvlist {-a}                # list all key/value pairs (-a all keys, including default)
    kvdump {-a}                # database dump (-a all keys, including default)
    kvimport                   # database import (overwrite)
    kvrestore                  # database restore (clear and restore)
    kvclear                    # clear database

Defaults database

kv-sh supports secondary read-only defaults database. If enabled, keys-value pairs from default value database are returned if local value is not specified.

Enable defaults database by setting DB_DEFAULTS_DIR:

DB_DIR="/tmp/.kv" DB_DEFAULTS_DIR="/tmp/.kv-default" . ./kv-sh
Improve this answer

Comments

-1

I ended up doing the following. Would prefer the variables in one file, but this bloats the code slightly. How does this read thing work? You can store multiple variables in a seperate file, say variables.txt, and then have your main program in say main.sh. It might be better to write seperate scripts for loading and saving variables though.

For varibles.txt:

A=0
B=0
C=0

For main.sh:

#!/bin/bash

#reload variables
A=`cat ./variables.txt|grep "A="|cut -d"=" -f2`
B=`cat ./variables.txt|grep "B="|cut -d"=" -f2`
C=`cat ./variables.txt|grep "C="|cut -d"=" -f2`

#print variables
printf "$A\n"
printf "$B\n"
printf "$C\n"

#update variables
A=$((($A+1)))
B=$((($B+2)))
C=$((($C+3)))

#save variables to file
#for A
 #remove entry for A
 cat ./variables.txt|grep -v "A=">>./tmp.txt
 #save entry for A
 printf "A=$A\n">>./tmp.txt
 #move tmp.txt to variables.txt
mv ./tmp.txt ./variables.txt

#for B
 #remove entry for B
 cat ./variables.txt|grep -v "B=">>./tmp.txt
 #save entry for B
 printf "B=$B\n">>./tmp.txt
 #move tmp.txt to variables.txt
 mv ./tmp.txt ./variables.txt

#for C
 #remove entry for C
 cat ./variables.txt|grep -v "C=">>./tmp.txt
 #save entry for C
 printf "C=$C\n">>./tmp.txt
 #move tmp.txt to variables.txt
 mv ./tmp.txt ./variables.txt
Improve this answer

1 Comment

No offense but this is terrible. You'd better source the file variables.txt instead of parsing it yourself. Besides, Useless Uses of Cat spotted!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.