2

I am building a model using mathopt library of or tools, and I want to save the model object as a pickle file (or any other suitable alternative). The use case for doing this is - underlying data behind the model build is huge, and it takes some time to build the model, however, when user interacts with the model, he/she will change only a couple of constraints at the maximum. So, I wanted to a) save the model object, and then b) load it back in memory, c) identify the constraint the user wants to change, and then update only that constraint, d) run the revised model and solve. Do the same process for multiple calls by user

However, pickling somehow is not working with mathopt:

from ortools.math_opt.python import mathopt

model = mathopt.Model(name="getting_started_lp")
x = model.add_variable(lb=0.0, ub=1.5, name="x")
y = model.add_variable(lb=0.0, ub=1.0, name="y")
model.add_linear_constraint(x + y <= 2)
model.maximize(x + 2 * y)

# try to pickle the model object
pickle.dumps(model)

# get the below error
AttributeError: Can't pickle local object 'WeakSet.__init__.<locals>._remove'

Pickle seems to work with cp-sat solver though:

model = cp_model.CpModel()
x = model.new_int_var(0, 10, "x")
y = model.new_int_var(0, 10, "y")
model.add(2 * x + 7 * y <= 20)
model.maximize(2 * x + 2 * y)
b = pickle.dumps(model)
# runs without any errors

Is there any way out - any alternative to saving the model for later use with mathopt ?

------ BELOW IS MY ATTEMPT TO SAVE THE MATHOPT MODEL -------- But does not work

from google.protobuf import text_format

from ortools.math_opt.python import mathopt

model = mathopt.Model(name="getting_started_lp")
x = model.add_variable(lb=0.0, ub=1.5, name="x")
y = model.add_variable(lb=0.0, ub=1.0, name="y")
model.add_linear_constraint(x + y <= 2)
model.maximize(x + 2 * y)

def export_model(model, filename):
    with open(filename, "w") as file:
        file.write(text_format.MessageToString(model.export_model()))


def import_model(filename):
    model = mathopt.Model()
    with open(filename, "r") as file:
        text_format.Parse(file.read(), model.export_model())
    return model

# runs fine -- exports model
export_model(model, "mdl")

# fails
import_model("mdl")

# ParseError: 1:11 : Message type # "operations_research.math_opt.ModelProto" should not have # multiple "objective" fields.
Improve this question
6
  • Why not use protobuf for CP-SAT? It is the native storage format. Commented Aug 2, 2024 at 13:18
  • Thanks Laurent. If you don't mind could you please show me how to do that through a simple example may be. Many thanks. Commented Aug 2, 2024 at 13:24
  • If you see my edit in the post, I have made an attempt to save the mathopt model, but clearly I am missing something. Hoping for your able guidance. Commented Aug 2, 2024 at 14:00
  • github.com/google/or-tools/blob/stable/ortools/sat/samples/… Commented Aug 2, 2024 at 16:17
  • 1
    Thanks, but I want to export a mathopt model to a file, which I can later import back. Could you please help me with the mathopt model pls. I tried to do in my code, but does not work. Thanks. Commented Aug 2, 2024 at 16:34

1 Answer 1

Reset to default
2

As Laurent said, I would recommend using proto instead of pickle generically whenever possible.

It appears that pickle won't work at all with mathopt due to the use of weak references.

MathOpt has the function in C++:

Model::FromModelProto()

How to pickel (save model object) a Google or-tool's mathopt model object

Which is what you need, but it is missing from the Python API.

A user wrote this code:

def from_model_proto(model_proto: model_pb2.ModelProto) -> mathopt.Model:
  """Converts a ModelProto of a Mixed-Integer Linear Program to a mathopt.Model.

  This functions builds a minimization problems with linear constraints and
  objective.

  Args:
    model_proto: The ModelProto to convert.

  Returns:
    The converted mathopt.Model.
  """
  model = mathopt.Model(name=model_proto.name)

  # Add variables.
  id_variable_map = {
      model_proto.variables.ids[i]: model.add_variable(
          name=model_proto.variables.names[i]
          if model_proto.variables.names
          else '',
          lb=model_proto.variables.lower_bounds[i],
          ub=model_proto.variables.upper_bounds[i],
          is_integer=model_proto.variables.integers[i],
      )
      for i, _ in enumerate(model_proto.variables.ids)
  }

  # Set objective.
  num_nonzeros_objective = len(model_proto.objective.linear_coefficients.ids)
  model.minimize(
      sum(
          id_variable_map[model_proto.objective.linear_coefficients.ids[i]]
          * model_proto.objective.linear_coefficients.values[i]
          for i in range(num_nonzeros_objective)
      )
  )

  # Set constraints.
  id_constraint_map = {
      model_proto.linear_constraints.ids[j]: model.add_linear_constraint(
          lb=model_proto.linear_constraints.lower_bounds[j],
          ub=model_proto.linear_constraints.upper_bounds[j],
          name=model_proto.linear_constraints.names[j],
      )
      for j, _ in enumerate(model_proto.linear_constraints.ids)
  }

  num_nonzero = len(model_proto.linear_constraint_matrix.column_ids)
  for i in range(num_nonzero):
    var_id = model_proto.linear_constraint_matrix.column_ids[i]
    con_id = model_proto.linear_constraint_matrix.row_ids[i]
    coef = model_proto.linear_constraint_matrix.coefficients[i]
    id_constraint_map[con_id].set_coefficient(id_variable_map[var_id], coef)

  return model

which probably does most of what you want (it doesn't handle as much of the ModelProto as the python API does, e.g. no quadratic objectives). You would need to read the proto from a file, then convert the proto back into a MathOpt model with this code.

A different (and complete) implementation will eventually be merged into mathopt, this is just a stopgap.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Thanks Ross for the prompt reply. This is very helpful. However, for my use case, I need to export my model as a file, and then later read it. So if I write the model_proto: ortools.math_opt.model_pb2.ModelProto to a file, then how I can read it back as a model_proto back ? At present it just reads it as a raw file. I am exporting model proto to local drive as follows file.write(text_format.MessageToString(model.export_model())) using from google.protobuf import text_format So the question is - how I can read it back in model_proto format in python. Rest is crystal clear.
@BhartenduAwasthi See protobuf docs on Writing and Reading / Python. The most important code-snippet is probably address_book.ParseFromString(f.read()) aka protobuf gives you a deserialization-function. Imho you should not make it text and keep files in binary (protobuf serialization is by default a binary-format). Don't let the name string fool you as protobuf just uses strings in many language-impls as type for storing RAW bytes aka binary (not displayable in terminal/editor !).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.