In my Django project, I am using Celery to asynchronously run my tasks.

The problem I am facing is that the "task_args" column in "django_celery_results_taskresult" is truncated (i.e., text followed by ellipsis).

For example, I am expecting the entire "process_4_items" value in "task_args." However, that is not the case. I can see the same truncated text in the command prompt of the Celery worker in execution (see below "[CELERY WORKER COMMAND PROMPT FOR ONE ITERATION]").

I really am not sure what I need to do to get the entire "process_4_items" value in "task_args."

I am detailing below everything I believe I need to reproduce this item.

Most important, when I debug the project and execute line by line, the entire content of "process_4_items" value appears in "task_args."

[celeryconfig.py]

broker_url = 'amqp://localhost:5672//'

imports = ('operation_hawkeye.tasks',)

result_backend = 'django-db'

accept_content = ['json']

task_always_eager = False

[views.py]

from celery import group

process_4_items = [

{

"service_id": "7f7962b082cb8da55e01f4f013d8fefbc97948ec03400c93423b050d",

"service_id_unique": "c30ed891987f4d8d3debcaef7edb0f582e6b634f1ebed00fb88f0ec7",

"service_type": "buzz_collect_compare",

"service_file_pf_folder": "",

"service_file_nanonsportsproductivity_folder": "",

"service_file_tba_report": "",

"service_file_buzz_collect_compare": "services/buzzcollect_xml_compare/2021/06/01/file_2_name_Update_COOKINGHD_May_2021_Listin_yWoWXo1.xml",

"service_file_type": "XML 1.0 document, ASCII text",

"service_file_name": "file_2_name_Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml",

"service_file_size": 396557,

"service_status": "In Queue for Upload",

"service_status_remarks": {

"source_information": {

"source_id": "5191",

"source_name": "10NEWS2",

"file_1_name": "Original_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml",

"file_2_name": "Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml"

}

},

"service_creation_date": "2021-06-01T18:18:40.968Z",

"service_owner": "[email protected]"

},

{

"service_id": "7f7962b082cb8da55e01f4f013d8fefbc97948ec03400c93423b050d",

"service_id_unique": "2edaf2535945992d13b9ee85e908b21cbd520a02642bd7faf7f229bc",

"service_type": "buzz_collect_compare",

"service_file_pf_folder": "",

"service_file_nanonsportsproductivity_folder": "",

"service_file_tba_report": "",

"service_file_buzz_collect_compare": "services/buzzcollect_xml_compare/2021/06/01/file_1_name_Original_COOKINGHD_May_2021_List_P2PlYY7.xml",

"service_file_type": "XML 1.0 document, ASCII text",

"service_file_name": "file_1_name_Original_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml",

"service_file_size": 396409,

"service_status": "In Queue for Upload",

"service_status_remarks": {

"source_information": {

"source_id": "5191",

"source_name": "10NEWS2",

"file_1_name": "Original_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml",

"file_2_name": "Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml"

}

},

"service_creation_date": "2021-06-01T18:18:40.935Z",

"service_owner": "[email protected]"

}

]

jobs_4 = group(tasks.buzz_collect_compare_main.s(process_4_items))

result_4 = jobs_4.apply_async()

[tasks.py]

@app.task(bind=True)

def buzz_collect_compare_main(self, items):

process_items = items

...

[CELERY WORKER COMMAND PROMPT FOR ONE ITERATION]

[2021-06-01 09:37:29,972: DEBUG/ForkPoolWorker-32] (0.010)

UPDATE

"django_celery_results_taskresult"

SET

"task_id" = 'daca83c1-3297-4e64-8a5e-f4b61aef84e2',

"task_name" = 'operation_hawkeye.tasks.buzz_collect_compare_main',

"task_args" = '([{''service_id'': ''dc30fc45948975044cb00c39a4d5a0db8e28bc34f0b4d52fd4dcddef'', ''service_id_unique'': ''2004e35ae5dcc2586ae5fca16ab8a5b2708ffeca233d3bb234feb347'', ''service_type'': ''buzz_collect_compare'', ''service_file_pf_folder'': '''', ''service_file_nanonsportsproductivity_folder'': '''', ''service_file_tba_report'': '''', ''service_file_buzz_collect_compare'': ''services/buzzcollect_xml_compare/2021/06/01/file_2_name_Update_COOKINGHD_May_2021_Listin_726wZgI.xml'', ''service_file_type'': ''XML 1.0 document, ASCII text'', ''service_file_name'': ''file_2_name_Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml'', ''service_file_size'': 396557, ''service_status'': ''In Queue for Upload'', ''service_status_remarks'': ''{"source_information": "{\"source_id\": \"27227\", \"source_name\": \"COOKINGHD\", \"file_1_name\": \"Original_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\", \"file_2_name\": \"Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\"}"}'', ''service_creation_date'': ''2021-06-01T09:36:56.825Z'', ''service_owner'': ''p...'', ...],)',

"task_kwargs" = '{}',

"status" = 'PROGRESS',

"worker" = 'celery@AMD-RYZEN-UBUNTU',

"content_type" = 'application/json',

"content_encoding" = 'utf-8',

"result" = '{"pending": false, "current": 1254, "total": 1254, "percent": 100.0, "description": "Building row 1254 of 1254"}',

"date_created" = '2021-06-01T09:36:58.465884+00:00'::timestamptz,

"date_done" = '2021-06-01T09:37:29.962267+00:00'::timestamptz,

"traceback" = NULL,

"meta" = '{"children": []}'

WHERE

"django_celery_results_taskresult"."id" = 808;

args=('daca83c1-3297-4e64-8a5e-f4b61aef84e2', 'operation_hawkeye.tasks.buzz_collect_compare_main', '([{\'service_id\': \'dc30fc45948975044cb00c39a4d5a0db8e28bc34f0b4d52fd4dcddef\', \'service_id_unique\': >\'2004e35ae5dcc2586ae5fca16ab8a5b2708ffeca233d3bb234feb347\', \'service_type\': \'buzz_collect_compare\', \'service_file_pf_folder\': \'\', \'service_file_nanonsportsproductivity_folder\': \'\', \'service_file_tba_report\': \'\', \'service_file_buzz_collect_compare\': \'services/buzzcollect_xml_compare/2021/06/01/file_2_name_Update_COOKINGHD_May_2021_Listin_726wZgI.xml\', \'service_file_type\': \'XML 1.0 document, ASCII text\', \'service_file_name\': \'file_2_name_Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\', \'service_file_size\': 396557, \'service_status\': \'In Queue for Upload\', \'service_status_remarks\': \'{"source_information": "{\\"source_id\\": \\"27227\\", \\"source_name\\": \\"COOKINGHD\\", \\"file_1_name\\": \\"Original_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\\", \\"file_2_name\\": \\"Update_COOKINGHD_May 2021 Listings(4-26-21 thru 5-30-21).xml\\"}"}\', \'service_creation_date\': \'2021-06-01T09:36:56.825Z\', \'service_owner\': \'p...\', ...],)', '{}', 'PROGRESS', 'celery@AMD-RYZEN-UBUNTU', 'application/json', 'utf-8', '{"pending": false, "current": 1254, "total": 1254, "percent": 100.0, "description": "Building row 1254 of 1254"}', datetime.datetime(2021, 6, 1, 9, 36, 58, 465884, tzinfo=<UTC>), datetime.datetime(2021, 6, 1, 9, 37, 29, 962267, tzinfo=<UTC>), '{"children": []}', 808)

[$ celery -A hawkeye report]

software -> celery:4.4.7 (cliffs) kombu:4.6.11 py:3.8.6

billiard:3.6.3.0 py-amqp:2.6.1

platform -> system:Linux arch:64bit, ELF

kernel version:5.8.0-53-generic imp:CPython

loader -> celery.loaders.app.AppLoader

settings -> transport:amqp results:django-db

accept_content: ['json']

broker_url: 'amqp://guest:********@localhost:5672//'

imports: ('operation_hawkeye.tasks',)

result_backend: 'django-db'

task_always_eager: False

[UPDATE]

Solved this issue by defining the argsrepr_maxsize value in celery.py. While I had known about this solution earlier, I did not realize that argsrepr_maxsize was within celery.ampq.

[celery.py]

import os

from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'hawkeye.settings')

os.environ.setdefault('FORKED_BY_MULTIPROCESSING', '1')

app = Celery('hawkeye')

default_config = 'hawkeye.celeryconfig'

app.config_from_object(default_config)

app.amqp.argsrepr_maxsize = 10485760