如何使用总线链将一项工作分成两个较小的零件？

Question

我对 cron 和工作有疑问。我的 cron 运行时间太长，因为数据太多。如何将代码分离为两个较小的任务。也许第一个任务可以处理数据，将它们保存在某个地方，下一个任务将在它们之间进行比较？此任务应该只记录一次购买中具有不同增值税的价目表。这是我第一次体验女巫链接，文档对我没有多大帮助。我很乐意提供任何建议:)

这是我的工作

<?php

namespace App\Jobs\Purchase;

use App\Models\Purchase;
use App\Jobs\Nodes\SendMail;
use Illuminate\Bus\Queueable;
use Illuminate\Support\Facades\Log;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

class VatValidation implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable;


    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct()
    {

    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        $purchases = Purchase::get();

        $errors = [];
        foreach ($purchases as $purchase) {
            $purchaseGroups = $purchase->purchasegroups()->get();

            if (!$purchaseGroups) // Purchase has no purchase groups
                continue;

            $purchaseGroupsTaxes = [];

            // cycle thru pricelists
            foreach ($purchaseGroups as $purchaseGroup) {
                $pricelists = $purchaseGroup->pricelists()->where('ordered', true)->get();

                if (!$pricelists)
                    continue;

                $purchaseGroupPricelistsTaxes = [];

                foreach ($pricelists as $pricelist) {

                    if (empty($pricelist->tax))
                        continue;

                    $taxWithoutDotComma = str_replace(['.', ','], '', $pricelist->tax);

                    array_push($purchaseGroupPricelistsTaxes, $taxWithoutDotComma);


                    if (count(array_unique($purchaseGroupPricelistsTaxes, SORT_REGULAR)) > 1)
                        // Log::info("Different taxes found for", ["pgroup" => $purchaseGroup, "pricelists" => $pricelists, "taxes" => $purchaseGroupPricelistsTaxes]);
                        array_push($errors, "Different taxes found for", ["pgroup" => $purchaseGroup, "pricelists" => $pricelists, "taxes" => $purchaseGroupPricelistsTaxes]);
                    else
                        array_push($purchaseGroupsTaxes, $taxWithoutDotComma); // Taxes are same we need just one value
                }

            }

            if (count(array_unique($purchaseGroupsTaxes, SORT_REGULAR)) > 1)
                Log::info("Different taxes in purchasegroups found for", ["purchase" => $purchase->name, "taxes" => $purchaseGroupsTaxes]);
                array_push($errors, "Different taxes in purchasegroups found for", ["purchase" => $purchase->name, "taxes" => $purchaseGroupsTaxes]);

        }

    }



}

这是 cron

<?php

namespace App\Jobs\Crons;

use Throwable;
use App\Models\Purchase;
use App\Jobs\Nodes\SendMail;
use App\Models\Purchasegroup;
use Illuminate\Bus\Queueable;
use App\Jobs\Traits\cronTrait;
use Illuminate\Support\Facades\Log;
use App\Jobs\Purchase\VatValidation;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Contracts\Queue\ShouldBeUnique;

class purchaseVatValidationCron implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, cronTrait;

    private $cronID;

    /**
     * Create a new job instance.
     *
     * @return void
     */
    public function __construct(int $cronID = null,)
    {
        $this->cronID = $cronID;

        $this->queue = 'longRunning';
    }

    /**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        VatValidation::dispatch()->onQueue('longRunning');


    }


    public function failed(Throwable $exception)
    {
        $this->saveLog($this->cronID, $exception->getMessage(), 'error');
        Log::error($exception->getMessage(), ["VatValidationCron error"]);
    }


}

我期待更有经验的人能给我一些如何解决问题的建议。

Answer 1

您应该正确优化它并确保它使用尽可能少的内存，而不是拆分您的工作，我无法想象您的工作会因为所有通过 Eloquent 调用

get

的循环而消耗多少内存。

您可以使用带有右连接的DB Facades Query（基于您在循环中过滤/跳过它们的方式），我不确定您的实际表结构和关系，但您可以执行如下查询。 对结果进行分块，最好将错误放入缓存中，以便您可以随时随地在应用程序中轻松检查它

public function handle() {

    // this just a made-up example base on how you do your loops, 
    // you need to update this query properly base on your table name and table relationship
    DB::table('purchases')
        ->rightJoin('purchase_groups as pg', 'purchases.id', '=', 'pg.purchase_id')
        ->rightJoin('price_lists as pl', 'pg.id', '=', 'pl.purchase_group_id')
        ->where('pl.ordered', true)
        ->whereNotNull('pl.tax')
        ->chunk(100, function (Collection $purchases)  {

            $errors = Cache::get('purchases_errors') ?? [];

            foreach ($purchases as $purchase) {
                // your logic here, 
                // all data from 3 tables would be on purchase var
                $errors[] = 'new error';
            }

            Cache::put('purchases_errors', $errors, now()->endOfDay() );
        });
}

Answer 2

在我看来，您只需要每个

PurchaseGroup

的不同税收值，而不真正需要任何其他数据，所以让我们简化一下。这将解决

n+1

性能问题并清理一下代码。

$purchaseGroupQuery = PurchaseGroup::whereHas('purchase')
    ->whereHas(['pricelist' => function ($query) {
        return $query->where('ordered', true)->whereNotNull('tax')
            ->selectRaw("REPLACE(
                REPLACE('tax', ',', ''), '.', ''
            ")->groupBy('tax');
    })->with('pricelist');

现在让我们进一步优化代码，使用

->chunk

来每

x

迭代释放内存（此时 x 为 25）。

$purchaseGroupQuery->chunk(25, function ($chunks) {
    $purchaseGroupTaxes = $purchaseGroup->pricelists->pluck('tax');

   // $purchaseGroupTaxes is now an array of unique tax values for the purchaselists within the given purchasegroup.
});

以上优化有：

1. 通过在循环中调用其他关系来消除嵌套

   n+1

   查询的需要
2. 不再需要检查循环内关系是否存在或为空
3. 提前为您格式化

   tax

   值
4. 确保返回的值对于每个

   unique

    都是 

   $purchaseGroupTaxes
5. 释放块之间的内存，以避免耗尽我们的跑步者。