I want to run my custom cron job, and I don’t know why it’s not registered or running.
In the database, I don’t see it registered—there’s no record of it.
In the log files, I don’t see any log messages from it, not even an error.
The code is correct—I’ve used it on an old website, and it was working. I even used it in a previous version of this website. This current version of the same website was hosted on shared hosting, and it was working. After moving to AWS, it’s not even registered or running.
What’s the issue?
I've tried everything I know: changed permissions, changed schedules in crontab.xml to every 5 minutes, cleared cache, upgraded the system—nothing worked.
I've waited for it to execute the previous two days since I set it to run every 24 hours, but it’s not running.
I’ve also used all the AI chatbots I can, but still no solution.
I really appreciate any suggestions or notes. If you have any, please don’t hesitate to provide them. 🙏
Here is a full overview of everything :
ubuntu@ip-172-31-38-33:/var/www/html/magento$ ls -l /var/www/html/magento/app/code/MetaCares/Chatbot
total 24
drwxr-xr-x 3 www-data www-data 4096 Mar 3 11:04 Controller
drwxr-xr-x 2 www-data www-data 4096 Mar 3 11:04 Cron
drwxr-xr-x 2 www-data www-data 4096 Mar 11 13:15 Model
drwxr-xr-x 4 www-data www-data 4096 Mar 21 10:27 etc
-rw-r--r-- 1 www-data www-data 170 Mar 3 11:04 registration.php
drwxr-xr-x 3 www-data www-data 4096 Mar 3 11:04 view
ubuntu@ip-172-31-38-33:/var/www/html/magento$ grep 'MetaCares_Chatbot' /var/www/html/magento/app/etc/config.php
'MetaCares_Chatbot' => 1,
ubuntu@ip-172-31-38-33:/var/www/html/magento$ sudo -u www-data /usr/bin/php8.3 /var/www/html/magento/bin/magento module:status MetaCares_Chatbot
MetaCares_Chatbot : Module is enabled
ubuntu@ip-172-31-38-33:/var/www/html/magento$ sudo cat /etc/crontab
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
SHELL=/bin/sh
# You can also override PATH, but by default, newer versions inherit it from the environment
#PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
17 * * * * root cd / && run-parts --report /etc/cron.hourly
25 6 * * * root test -x /usr/sbin/anacron || { cd / && run-parts --report /etc/cron.daily; }
47 6 * * 7 root test -x /usr/sbin/anacron || { cd / && run-parts --report /etc/cron.weekly; }
52 6 1 * * root test -x /usr/sbin/anacron || { cd / && run-parts --report /etc/cron.monthly; }
#
ubuntu@ip-172-31-38-33:/var/www/html/magento$ crontab -l
#~ MAGENTO START f37deed947b2ea951ad6f939b8ab752bc79587e3d77f40d06f20f0657c98e94d
* * * * * www-data /usr/bin/php8.3 /var/www/html/magento/bin/magento cron:run 2>&1 | grep -v "Ran jobs by schedule" >> /var/www/html/magento/var/log/magento.cron.log
* * * * * www-data /usr/bin/php8.3 /var/www/html/magento/update/cron.php
* * * * * www-data /usr/bin/php8.3 /var/www/html/magento/bin/magento setup:cron:run
#~ MAGENTO END f37deed947b2ea951ad6f939b8ab752bc79587e3d77f40d06f20f0657c98e94d
ubuntu@ip-172-31-38-33:/var/www/html/magento$ sudo -u www-data /usr/bin/php8.3 /var/www/html/magento/bin/magento cron:install
Crontab has already been generated and saved
ubuntu@ip-172-31-38-33:/var/www/html/magento$ sudo -u www-data /usr/bin/php8.3 /var/www/html/magento/bin/magento cache:clean
Cleaned cache types:
config
layout
block_html
collections
reflection
db_ddl
compiled_config
eav
customer_notification
config_integration
config_integration_api
full_page
config_webservice
translate
ubuntu@ip-172-31-38-33:/var/www/html/magento$ sudo -u www-data /usr/bin/php8.3 /var/www/html/magento/bin/magento cron:run
Ran jobs by schedule.
ubuntu@ip-172-31-38-33:/var/www/html/magento$ sudo -u www-data /usr/bin/php8.3 /var/www/html/magento/bin/magento cron:run
Ran jobs by schedule.
ubuntu@ip-172-31-38-33:/var/www/html/magento$ mysql -u my-username -p my-database
Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 812095
Server version: 8.0.41-0ubuntu0.24.04.1 (Ubuntu)
Copyright (c) 2000, 2025, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> SELECT DISTINCT job_code FROM cron_schedule;
+-----------------------------------------------+
| job_code |
+-----------------------------------------------+
| amasty_base_instance_registration |
| amasty_cron_activity |
| bulk_cleanup |
| bulk_mark_incomplete_operations_as_failed |
| captcha_delete_expired_images |
| captcha_delete_old_attempts |
| catalog_index_refresh_price |
| catalog_product_attribute_value_synchronize |
| catalog_product_frontend_actions_flush |
| catalog_product_outdated_price_values_cleanup |
| consumers_runner |
| core_process_feed |
| expired_tokens_cleanup |
| indexer_clean_all_changelogs |
| indexer_reindex_all_invalid |
| indexer_update_all_views |
| magento_newrelicreporting_cron |
| messagequeue_clean_outdated_locks |
| newsletter_send_all |
| outdated_authentication_failures_cleanup |
| sales_clean_orders |
| sales_grid_order_async_insert |
| sales_grid_order_creditmemo_async_insert |
| sales_grid_order_invoice_async_insert |
| sales_grid_order_shipment_async_insert |
| sales_send_order_creditmemo_emails |
| sales_send_order_emails |
| sales_send_order_invoice_emails |
| sales_send_order_shipment_emails |
| security_deactivate_expired_users |
| stripe_capture_authorizations |
| stripe_webhooks_configure |
| stripe_webhooks_ping |
| stripe_webhook_cleanup_tables |
| stripe_webhook_events_retry |
+-----------------------------------------------+
35 rows in set (0.00 sec)
mysql> exit
Bye
system.log file content:
[2025-03-21T11:04:07.138051+00:00] main.ERROR: crontab -l 2>/dev/null 2>&1
no crontab for www-data [] []
[2025-03-21T11:04:07.140750+00:00] main.ERROR: crontab -l 2>/dev/null 2>&1
no crontab for www-data [] []
[2025-03-21T11:04:07.146243+00:00] main.INFO: echo "
#~ MAGENTO START f37deed947b2ea951ad6f939b8ab752bc79587e3d77f40d06f20f0657c98e94d
* * * * * /usr/bin/php8.3 /var/www/html/magento/bin/magento cron:run 2>&1 | grep -v \"Ran jobs by schedule\" >> /var/www/html/magento/var/log/magento.cron.log
#~ MAGENTO END f37deed947b2ea951ad6f939b8ab752bc79587e3d77f40d06f20f0657c98e94d
" 2>&1 | crontab - 2>&1
[] []
[2025-03-21T11:14:42.345633+00:00] main.INFO: crontab -l 2>/dev/null 2>&1
#~ MAGENTO START f37deed947b2ea951ad6f939b8ab752bc79587e3d77f40d06f20f0657c98e94d
* * * * * /usr/bin/php8.3 /var/www/html/magento/bin/magento cron:run 2>&1 | grep -v "Ran jobs by schedule" >> /var/www/html/magento/var/log/magento.cron.log
#~ MAGENTO END f37deed947b2ea951ad6f939b8ab752bc79587e3d77f40d06f20f0657c98e94d
[] []
cron.log file content:
[2025-03-21T11:14:02.740900+00:00] main.INFO: Cron Job indexer_reindex_all_invalid is run [] []
[2025-03-21T11:14:02.749678+00:00] main.INFO: Cron Job indexer_reindex_all_invalid is successfully finished. Statistics: {"sum":0.0089640617370605,"count":1,"realmem":0,"emalloc":456448,"realmem_start":189267968,"emalloc_start":177734544} [] []
[2025-03-21T11:14:02.775097+00:00] main.INFO: Cron Job indexer_update_all_views is run [] []
[2025-03-21T11:14:02.779181+00:00] main.INFO: Cron Job indexer_update_all_views is successfully finished. Statistics: {"sum":0.0040781497955322,"count":1,"realmem":0,"emalloc":115968,"realmem_start":189267968,"emalloc_start":178193200} [] []
[2025-03-21T11:14:02.919780+00:00] main.INFO: Cron Job consumers_runner is run [] []
[2025-03-21T11:14:02.927706+00:00] main.INFO: Cron Job consumers_runner is successfully finished. Statistics: {"sum":0.008112907409668,"count":1,"realmem":0,"emalloc":188096,"realmem_start":189267968,"emalloc_start":178074696} [] []
.... and alot of other messages but my cron job message not appearing in the message list
Location of crontab.xml in Magento Module
magento
└── app
└── code
└── MetaCares
└── Chatbot
└── etc
├── acl.xml
├── adminhtml
│ └── ... (folders inside adminhtml)
├── crontab.xml
├── frontend
│ └── ... (folders inside frontend)
└── module.xml
crontab.xml:
<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Cron:etc/crontab.xsd">
<group id="default">
<job name="chatbot_update_products" instance="MetaCares\Chatbot\Cron\UpdateProducts"
method="execute">
<schedule>0 0 * * *</schedule>
</job>
</group>
</config>
Location of UpdateProducts.php in Magento Module
magento
└── app
└── code
└── MetaCares
└── Chatbot
└── etc
| ├── acl.xml
| ├── adminhtml
| │ └── ...
| ├── crontab.xml
| ├── frontend
| │ └── ...
| └── module.xml
└── Cron
| ├── UpdateProducts.php
UpdateProducts.php:
<?php
namespace MetaCares\Chatbot\Cron;
use Psr\Log\LoggerInterface;
use Magento\Framework\App\ObjectManager;
use Magento\Catalog\Model\ResourceModel\Product\CollectionFactory;
use Magento\Catalog\Model\CategoryRepository;
use Magento\Framework\Encryption\EncryptorInterface;
use Magento\Framework\App\Config\ScopeConfigInterface;
class UpdateProducts
{
protected $logger;
private $productCollectionFactory;
private $categoryRepository;
private $weaviateEndpoint;
private $weaviateApiKey;
private $scopeConfig;
private $encryptor;
private $authorization;
public function __construct(
LoggerInterface $logger,
CollectionFactory $productCollectionFactory,
CategoryRepository $categoryRepository,
ScopeConfigInterface $scopeConfig,
EncryptorInterface $encryptor
) {
$this->logger = $logger;
$this->productCollectionFactory = $productCollectionFactory;
$this->categoryRepository = $categoryRepository;
$this->scopeConfig = $scopeConfig;
$this->encryptor = $encryptor;
$this->initializeApiKeys();
}
private function initializeApiKeys()
{
$this->weaviateEndpoint = $this->scopeConfig->getValue('chatbot/weaviate/endpoint', \Magento\Store\Model\ScopeInterface::SCOPE_STORE);
$this->weaviateApiKey = $this->scopeConfig->getValue('chatbot/weaviate/api_key', \Magento\Store\Model\ScopeInterface::SCOPE_STORE);
$this->authorization = $this->scopeConfig->getValue('chatbot/openai/authorization', \Magento\Store\Model\ScopeInterface::SCOPE_STORE);
try {
$this->weaviateApiKey = $this->encryptor->decrypt($this->weaviateApiKey);
$this->authorization = $this->encryptor->decrypt($this->authorization);
} catch (\Exception $e) {
$this->logger->error("Error decrypting API keys: " . $e->getMessage());
$this->weaviateApiKey = '';
}
}
/**
* Fetch products from Magento.
*/
public function fetchProducts(): array
{
try {
$productCollection = $this->productCollectionFactory->create();
$productCollection->addAttributeToSelect([
'name',
'sku',
'price',
'description',
'short_description',
'category_ids'
]);
$products = [];
foreach ($productCollection as $product) {
$products[] = [
'name' => $product->getName(),
'sku' => $product->getSku(),
'price' => $product->getPrice(),
'description' => $product->getDescription() ?? '',
'short_description' => $product->getShortDescription() ?? '',
'categories' => $this->getCategoryNames($product->getCategoryIds()),
'product_url' => $product->getUrlModel()->getUrl($product),
];
}
if (count($products) > 0) {
$this->logger->info("Fetched " . count($products) . " products from Magento.");
} else {
$this->logger->info("NO Products are fetched from magento.");
}
return $products;
} catch (\Exception $e) {
$this->logger->error('Error fetching products: ' . $e->getMessage());
$this->logger->error((string)$e);
return [];
}
}
/**
* Push products to Weaviate, updating if they exist.
*/
private function pushProductsToWeaviate(array $products): array
{
try {
$results = [];
foreach ($products as $product) {
try {
$data = [
"class" => "Product",
"properties" => [
"name" => $product['name'],
"sku" => $product['sku'],
"price" => (float) $product['price'],
"description" => $product['description'],
"short_description" => $product['short_description'],
"categories" => $product['categories'],
"product_url" => $product['product_url']
]
];
$existingProduct = $this->getProductBySku($product['sku']);
if ($existingProduct) {
$results[] = $this->updateProductInWeaviate($existingProduct['_additional']['id'], $data);
} else {
$results[] = $this->insertProductToWeaviate($data);
}
} catch (\Exception $e) {
$this->logger->error('Error processing product with SKU: ' . $product['sku'] . 'Error message : ' . $e->getMessage());
$this->logger->error((string)$e);
}
}
return $results;
} catch (\Exception $e) {
$this->logger->error('Error push products to Weaviate : ' . $e->getMessage());
$this->logger->error((string)$e);
return [];
}
}
/**
* Get product by SKU from Weaviate.
*/
private function getProductBySku(string $sku): ?array
{
$query = [
"query" => "{
Get {
Product(where: {
path: [\"sku\"],
operator: Equal,
valueString: \"$sku\"
}) {
_additional {
id
}
name
sku
price
description
short_description
categories
product_url
}
}
}"
];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $this->weaviateEndpoint . '/v1/graphql');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($query));
curl_setopt($ch, CURLOPT_HTTPHEADER, [
'Content-Type: application/json',
'Authorization: Bearer ' . $this->weaviateApiKey,
'X-Openai-Api-Key: ' . $this->authorization,
]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
$error = curl_error($ch);
curl_close($ch);
if ($error) {
$this->logger->error("Error getting product by SKU ($sku): " . $error);
return null;
}
$result = json_decode($response, true);
return $result['data']['Get']['Product'][0] ?? null;
}
/**
* Insert a new product into Weaviate.
*/
private function insertProductToWeaviate(array $data): array
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $this->weaviateEndpoint . '/v1/objects');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($data));
curl_setopt($ch, CURLOPT_HTTPHEADER, [
'Content-Type: application/json',
'Authorization: Bearer ' . $this->weaviateApiKey,
'X-Openai-Api-Key: ' . $this->authorization,
]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
$error = curl_error($ch);
curl_close($ch);
if ($error) {
$this->logger->error("Error inserting product into Weaviate: " . $error);
return [];
}
return json_decode($response, true);
}
/**
* Update an existing product in Weaviate.
*/
private function updateProductInWeaviate(string $id, array $data): array
{
$deleteUrl = $this->weaviateEndpoint . "/v1/objects/Product/$id";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $deleteUrl);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "DELETE");
curl_setopt($ch, CURLOPT_HTTPHEADER, [
'Content-Type: application/json',
'Authorization: Bearer ' . $this->weaviateApiKey,
'X-Openai-Api-Key: ' . $this->authorization,
]);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$deleteResponse = curl_exec($ch);
$deleteError = curl_error($ch);
$deleteHttpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if ($deleteError) {
$this->logger->error("Error deleting product in Weaviate with ID ($id): " . $deleteError);
return ['error' => $deleteError];
}
if ($deleteHttpCode !== 204) {
$this->logger->error("Failed to delete product in Weaviate. HTTP Code: $deleteHttpCode, Response: $deleteResponse");
return ['error' => "HTTP Code: $deleteHttpCode, Response: $deleteResponse"];
}
$insertData = [
"class" => "Product",
"properties" => $data['properties']
];
$insertResponse = $this->insertProductToWeaviate($insertData);
if (isset($insertResponse['error'])) {
$this->logger->error("Error inserting product into Weaviate: " . json_encode($insertResponse['error']));
return $insertResponse;
}
return $insertResponse;
}
/**
* Get category names by IDs.
*/
private function getCategoryNames(array $categoryIds): array
{
try {
$categoryNames = [];
foreach ($categoryIds as $categoryId) {
try {
$category = $this->categoryRepository->get($categoryId);
$categoryNames[] = $category->getName();
} catch (\Exception $e) {
$this->logger->error('Error getCategoryNames product with categoryId: ' . $categoryId . 'Error message : ' . $e->getMessage());
$this->logger->error((string)$e);
continue;
}
}
return $categoryNames;
} catch (\Exception $e) {
$this->logger->error('Error get category names: ' . $e->getMessage());
$this->logger->error((string)$e);
return [];
}
}
public function execute()
{
$this->logger->info('Cron job for updating product data in Weaviate is starting');
try {
$products = $this->fetchProducts();
if (!empty($products)) {
$results = $this->pushProductsToWeaviate($products);
}
$this->logger->info('Cron job for updating product data in Weaviate is finished.');
} catch (\Exception $e) {
$this->logger->error('Cron job for updating product data in Weaviate failed: ' . $e->getMessage());
$this->logger->error((string)$e);
}
}
}
