How to load multiple dependencies in py3_image bazel rule

Question

I recently started working with bazel so admittedly, have little knowledge of bazel intricacies. I'm using bazel to generate docker images but I want to use multiple deps inside the py3_image rule.

I have a BUILD.bazel which has python rule as follows:

load("@io_bazel_rules_docker//python3:image.bzl", "py3_image")
load("@io_bazel_rules_docker//container:container.bzl", "container_push")
load("@custom_deps//:requirements.bzl", "requirement")

exports_files(["component.yaml"])

py3_image(
    name = "custom",
    srcs = [
        "src/payload_populator/bq_populator.py",
        "src/payload_populator/cloudsql_fetcher.py",
        "src/payload_populator/config.py",
        "src/payload_populator/SingleListing.py",
        "src/payload_populator/sql.py",
        "src/custom/browse.py",
        "src/custom/closet.py",
        "src/custom/constants.py",
        "src/custom/listing.py",
        "src/custom/util.py",
        "src/session/session.py"
    ],
    base = "@python//image",
    main = "src/payload_populator/bq_populator.py",
    visibility = ["//visibility:public"],
    deps = [
        requirement("google-cloud-bigquery"),
        requirement("google-cloud-core"),
        "//common:common_lib",
    ],
)

# https://github.com/bazelbuild/rules_docker/tree/e15c9ebf203b7fa708e69ff5f1cdcf427d7edf6f#container_push
container_push(
    name = "push_custom",
    format = "Docker",
    image = ":custom",
    registry = "gcr.io",
    repository = "rental-ds/custom",
    tag = "$(BRANCH_NAME)",
)

I have 120+ dependencies that my code relies on inside

deps = [
        requirement("google-cloud-bigquery"),
        requirement("google-cloud-core"),
        "//common:common_lib",
    ],

I don't want to list out all of them independently to use them in the code. Is there a simple way to either import all of them in one go from requirement or a way to bypass my calling of requirement("library")?

I've tried to scour Bazel docs: https://docs.bazel.build/versions/main/be/python.html

and the github page for docker-rules: https://github.com/bazelbuild/rules_docker

If I'm missing some knowledge that's obvious, please link a reference for the read as well.

A better solution IMO would be to to make multiple py_libraries (possibly even one per .py source file) and specify their requirements. Then you can depend on them and don't need to specify the transitive deps again. — Laurenz
– Laurenz, Commented Jan 29, 2022 at 10:53

SG_Bazel · Accepted Answer · 2023-01-04 06:05:53Z

0

It can be achieved with bazel.build/reference/be/functions#glob function. Have you tried. Seems old issue but wanted to see.

answered Jan 4, 2023 at 6:05

SG_Bazel

3532 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

msudder · Accepted Answer · 2023-07-14 16:47:52Z

0

having BUILD.bazel file in each package makes for a convenient place to express unit-tests close to the code and then srcs becomes only local files and deps gets to be depending on your packages:

py_binary(
    name = "my_app",
    srcs = glob(["*.py", "src/my_app/*.py"]),
    deps = ["src/payload_populator", "src/custom"],
)

py_image(
    name = "app_image",
    deps = [":my_app"],
    cmd = ["./my-app"]
)

answered Jul 14, 2023 at 16:47

msudder

5123 silver badges14 bronze badges

Collectives™ on Stack Overflow

How to load multiple dependencies in py3_image bazel rule

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related