PHP 生成器对象未从已处理的外部 JSON API 接收数据

问题描述 投票：0回答：1

我一直使用此资源作为参考，将基于 Joomla 3 的 CLI 脚本转换为基于 Joomla 4/5 的 API 脚本。该脚本从外部 API 检索新闻帖子，并将它们作为单独的文章添加到 Joomla。

我的 

$process
功能可以正常工作。使用
print_r
函数时，成功连接外部API并在浏览器中输出数据数组。我已经在下面包含了完整的 
$process
功能。

$process = function (string $givenHttpVerb, string $endPoint, string $dataString, array $headers, int $timeOut, $transport) {
  curl_setopt_array($transport, [
      CURLOPT_URL            => $endPoint,
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_ENCODING       => 'utf-8',
      CURLOPT_MAXREDIRS      => 10,
      CURLOPT_TIMEOUT        => $timeOut,
      CURLOPT_FOLLOWLOCATION => true,
      CURLOPT_SSLVERSION     => CURL_SSLVERSION_TLSv1_2,
      CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_2TLS,
      CURLOPT_CUSTOMREQUEST  => $givenHttpVerb,
      CURLOPT_POSTFIELDS     => $dataString,
      CURLOPT_HTTPHEADER     => $headers,
    ]
  );

  $response = curl_exec($transport);

  if (empty($response)) {
    throw new RuntimeException( 'Empty output', 422 );
  }
  return $response;
};

使用 

$process
 调用
$dataSourceResponse = $process($dataSourceHttpVerb, $dataSourceUrl, $dataSourceDataString, $dataSourceHeaders, $dataSourceTimeout, $dataSourceTransport);

函数

我有一个 

$generator
函数，它接受 
$dataSourceResponse
匿名函数和 
$apiUrl
，用于基于 Joomla 的网站。 
$apiUrl
是 upgrade.domain.com/api/index.php/v1。下面包含完整的 
$generator
功能。

$generator = function (string $dataSourceResponse): Generator {
  if (empty($dataSourceResponse)) {
    yield new RuntimeException( 'DTN API response must not be empty', 422 );
  }

  $resource = json_decode($dataSourceResponse);

  if ($resource === false) {
    yield new RuntimeException( 'Could not read response from source DTN API', 500 );
  }

  try {
    foreach ($resource as $article) {
      $data = [
        'id' => 0,
        'catid' => 13,
        'title' => $article->title,
        'articletext' => $article->content,
        'introtext' => $article->storySummary,
        'fulltext' => $article->content,
        'note' => $article->topic,
        'state' => 1,
        'access' => 1,
        'created_by' => 386,
        'created_by_alias' => 'DTN News',
        'language' => '*',
      ];
    }
    $dataString = json_encode($data);
  } finally {
    echo 'Done processing data' . PHP_EOL;
  }
};

使用 

$generator
 调用
$postData = $generator($dataSourceResponse, $apiUrl);

函数

当我使用 

print_r
作为 
$postData
变量时，浏览器中会显示以下内容：
Generator Object() Done processing data
。

在我看来，从外部 API 检索到的数据并未成功与用于通过 API 将新闻文章插入 Joomla 文章的 

$generator
函数共享。

我在下面提供了完整的 PHP 脚本以供参考。

declare(strict_types=1);
ini_set('error_reporting', E_ALL & ~E_DEPRECATED);

$dataSourceUrl = 'https://api.dtn.com/publishing/news/articles?categoryId=1%2C2%2C3%2C4%2C5%2C6%2C16%2C17&limit=10&maxAge=15&apikey=redacted';
$apiUrl = 'https://upgrade.domain.com/api/index.php/v1';
$token = redacted;
$timeOut = 60;

$generator = function (string $dataSourceResponse): Generator {
  if (empty($dataSourceResponse)) {
    yield new RuntimeException( 'DTN API response must not be empty', 422 );
  }

  $resource = json_decode($dataSourceResponse);

  if ($resource === false) {
    yield new RuntimeException( 'Could not read response from source DTN API', 500 );
  }

  try {
    foreach ($resource as $article) {
      $data = [
        'id' => 0,
        'catid' => 13,
        'title' => $article->title,
        'articletext' => $article->content,
        'introtext' => $article->storySummary,
        'fulltext' => $article->content,
        'note' => $article->topic,
        'state' => 1,
        'access' => 1,
        'created_by' => 386,
        'created_by_alias' => 'DTN News',
        'language' => '*',
      ];
    }
    $dataString = json_encode($data);
  } finally {
    echo 'Done processing data' . PHP_EOL;
  }
};

$process = function (string $givenHttpVerb, string $endPoint, string $dataString, array $headers, int $timeOut, $transport) {
  curl_setopt_array($transport, [
      CURLOPT_URL            => $endPoint,
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_ENCODING       => 'utf-8',
      CURLOPT_MAXREDIRS      => 10,
      CURLOPT_TIMEOUT        => $timeOut,
      CURLOPT_FOLLOWLOCATION => true,
      CURLOPT_SSLVERSION     => CURL_SSLVERSION_TLSv1_2,
      CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_2TLS,
      CURLOPT_CUSTOMREQUEST  => $givenHttpVerb,
      CURLOPT_POSTFIELDS     => $dataString,
      CURLOPT_HTTPHEADER     => $headers,
    ]
  );

  $response = curl_exec($transport);

  if (empty($response)) {
    throw new RuntimeException( 'Empty output', 422 );
  }
  return $response;
};

$dataSourceHttpVerb = 'GET';
$dataSourceDataString = '';
$dataSourceHeaders = [
  'Accept: application/json',
  'Accept-Encoding: deflate, gzip, br',
  'Content-Type: application/json',
  'Connection: keep-alive',
];
$dataSourceTimeout = 60;
$dataSourceTransport = curl_init();

try {
  $dataSourceResponse = $process($dataSourceHttpVerb, $dataSourceUrl,   $dataSourceDataString, $dataSourceHeaders, $dataSourceTimeout, $dataSourceTransport);
  $postData = $generator($dataSourceResponse, $apiUrl);

  foreach ($postData as $dataString) {
    if (!is_string($dataString)) {
      continue;
    }
    $curl = curl_init();
    try {
      $headers = [
        'Accept: application/vnd.api+json',
        'Content-Type: application/json',
        'Content-length: ' . mb_strlen($dataString),
        sprintf('X-Joomla-Token: %s', trim($token)),
      ];
  
      $output = $process('POST', $apiUrl, $dataString, $headers, $timeOut, $curl);
  
    } catch (Throwable $e) {
      echo $e->getMessage() . PHP_EOL;
      continue;
    } finally {
      curl_close($curl);
    }
  }
} catch (Throwable $e) {
  echo $e->getMessage() . PHP_EOL;
} finally {
  curl_close($dataSourceTransport);
}

这里是来自基于 CLI 的脚本的 PHP 

foreach loop
，我已尝试对其进行调整以供参考。

foreach ($articles as $article) {
  $articleData = [
    'id' => 0,
    'catid' => 13,
    'title' => $article->title,
    'introtext' => $article->storySummary,
    'fulltext' => $article->content,
    'note' => $article->topic,
    'state' => 1,
    'access' => 1,
    'created_by' => 386,
    'created_by_alias' => 'DTN News',
    'language' => '*',
  ];

  if (!$articleModel->save($articleData)) {
    throw new Exception($articleModel->getError());
  }
}

更新

在数据数组上使用 

var_dump(json_encode($data)); die();
的输出：

{
  "id": 0,
  "catid": 13,
  "title": "DTN Midday Livestock Comments",
  "articletext": "Firm gains redeveloped early Wednesday morning in live cattle and feeder cattle trade. This support has helped to spark some underlying follow-through buying in nearby and deferred contracts, supporting triple-digit gains through the morning. Hog futures are lightly traded but holding narrow losses at midday based on softness in pork fundamentals.<\\/span>",
  "introtext": "Firm gains redeveloped early Wednesday morning in live cattle and feeder cattle trade. This support has helped to spark some underlying follow-through buying in nearby and deferred contracts, supporting triple-digit gains through the morning. Hog futures are lightly traded but holding narrow losses at midday based on softness in pork fundamentals.<\\/span>",
  "fulltext": "Firm gains redeveloped early Wednesday morning in live cattle and feeder cattle trade. This support has helped to spark some underlying follow-through buying in nearby and deferred contracts, supporting triple-digit gains through the morning. Hog futures are lightly traded but holding narrow losses at midday based on softness in pork fundamentals.<\\/span>",
  "note": "DTN\\/Ag\\/Markets",
  "state": 1,
  "access": 1,
  "created_by": 386,
  "created_by_alias": "DTN News",
  "language": "*"
}
php json curl joomla
1个回答
0
投票

我通过重构代码并实现我用作此脚本基础的资源中缺少的一些功能来解决了我的问题。我在下面突出显示了重构的代码和新实现的代码以及完整的脚本以供参考。

通过将 

$data foreach loop
移动到循环内部来重构 
yield (json_encode($data));
。我还添加了字符串操作函数来清理一些导入的文本内容。

// Strip CSS classes from tags without removing the tags themselves
$stripClasses = '/\bclass=["\'][^"\']*["\']/';

foreach ($resource as $article) {
  $data = [
    'id' => 0,
    'catid' => 13,
    'title' => ucwords(strtolower($article->title)),
    'alias' => strtolower(str_replace(' ', '-', urlencode($article->title))),
    'articletext' => $article->content,
    'introtext' => $article->storySummary,
    'fulltext' => $article->content,
    'note' => 'DTN supplied topics: ' . str_replace('/', ', ', $article->topic),
    'state' => 1,
    'access' => 1,
    'created_by' => 386,
    'created_by_alias' => 'DTN News',
    'language' => '*',
  ];
  
  if (isset($data['articletext']) && !empty($data['articletext'])) {
    $data['articletext'] = strip_tags($data['articletext'], $allowedTags);
    $data['articletext'] = preg_replace($stripClasses, '', $data['articletext']);
  } else {
    $data['articletext'] = '';
  }
  if (isset($data['introtext']) && !empty($data['introtext'])) {
    $data['introtext'] = strip_tags($data['introtext'], $allowedTags);
    $data['introtext'] = preg_replace($stripClasses, '', $data['introtext']);
  } else {
    $data['introtext'] = '';
  }
  if (isset($data['fulltext']) && !empty($data['fullotext'])) {
    $data['fulltext'] = strip_tags($data['fulltext'], $allowedTags);
    $data['fulltext'] = preg_replace($stripClasses, '', $data['fulltext']);
  } else {
    $data['fulltext'] = '';
  }
  
  if ($data === false) {
  } else {
    yield (json_encode(
      $data
    ));
  }
}

我从资源中包含了这个匿名函数来处理外部和 Joomla 端点。这对于解决我在重构代码时遇到的 

404 Resource not found
错误至关重要。

$endPoint = function (string $givenBaseUrl, string $givenBasePath, int $givenResourceId = 0): string {
  return $givenResourceId ? sprintf('%s/%s/%s/%d', $givenBaseUrl, $givenBasePath, 'content/articles', $givenResourceId)
: sprintf('%s/%s/%s', $givenBaseUrl, $givenBasePath, 'content/articles');
};

我从 

$apiUrl
匿名函数中删除了 
$postData
变量，因为它没有必要。

$postData = $generator($dataSourceResponse);

最终 

foreach loop
中的第一个 
try > catch > finally
进行了重构，以检查 API 的 
PATCH
或 
POST
并处理新包含的 
$endPoint
匿名函数。

$storage = [];
foreach ($postData as $dataString) {
  if (!is_string($dataString)) {
    continue;
  }
  $curl = curl_init();
  try {
    $decodedDataString = json_decode($dataString);
    if ($decodedDataString === false) {
      continue;
    }
  
    $headers = [
      'Accept: application/vnd.api+json',
      'Content-Type: application/json',
      'Content-length: ' . mb_strlen($dataString),
      sprintf('X-Joomla-Token: %s', trim($token)),
    ];
    $pk = (int) $decodedDataString->id;
    $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $curl);
  
    $decodedJsonOutput = json_decode($output);
  
    if (isset($decodedJsonOutput->errors)) {
      $storage[] = ['mightExists' => $decodedJsonOutput->errors[0]->code === 400, 'decodedDataString' => $decodedDataString];
      continue;
    }
  
  } catch (Throwable $e) {
    echo $e->getMessage() . PHP_EOL;
    continue;
  } finally {
    curl_close($curl);
  }
}

添加第二个 

foreach loop
来处理错误并检查 Joomla 文章的重复别名。

foreach ($storage as $item) {
  $storageCurl = curl_init();
  try {
    if ($item['mightExists']) {
      $pk = (int) $item['decodedDataString']->id;
      $item['decodedDataString']->alias = sprintf('%s-%s', $item['decodedDataString']->alias, bin2hex(random_bytes(4)));
      // No need to do another json_encode anymore
      $dataString = json_encode($item['decodedDataString']);
      // HTTP request headers
      $headers = [
        'Accept: application/vnd.api+json',
        'Content-Type: application/json',
        'Content-Length: ' . mb_strlen($dataString),
        sprintf('X-Joomla-Token: %s', trim($token)),
      ];
      $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $storageCurl);
    }
  } catch (Throwable $storageThrowable) {
    echo $storageThrowable->getMessage() . PHP_EOL;
    continue;
  } finally {
    curl_close($storageCurl);
  }
}

完整脚本

declare(strict_types=1);
// ini_set('error_reporting', E_ALL & ~E_DEPRECATED);

$dataSourceUrl = Redacted;

$baseUrl= Redacted;
$basePath = 'api/index.php/v1';
$token = Redacted;

$timeOut = 120;

$generator = function (string $dataSourceResponse): Generator {
  if (empty($dataSourceResponse)) {
    yield new RuntimeException( 'DTN API response must not be empty', 422 );
  }

  $resource = json_decode($dataSourceResponse);

  if ($resource === false) {
    yield new RuntimeException( 'Could not read response from source DTN API', 500 );
  }

  $allowedTags = ['p', 'img', 'table'];
  $stripClasses = '/\bclass=["\'][^"\']*["\']/';

  try {
    foreach ($resource as $article) {
      $data = [
        'id' => 0,
        'catid' => 13,
        'title' => ucwords(strtolower($article->title)),
        'alias' => strtolower(str_replace(' ', '-', urlencode($article->title))),
        'articletext' => $article->content,
        'introtext' => $article->storySummary,
        'fulltext' => $article->content,
        'note' => 'DTN supplied topics: ' . str_replace('/', ', ', $article->topic),
        'state' => 1,
        'access' => 1,
        'created_by' => 386,
        'created_by_alias' => 'DTN News',
        'language' => '*',
      ];
  
      if (isset($data['articletext']) && !empty($data['articletext'])) {
        $data['articletext'] = strip_tags($data['articletext'], $allowedTags);
        $data['articletext'] = preg_replace($stripClasses, '', $data['articletext']);
      } else {
        $data['articletext'] = '';
      }
      if (isset($data['introtext']) && !empty($data['introtext'])) {
        $data['introtext'] = strip_tags($data['introtext'], $allowedTags);
        $data['introtext'] = preg_replace($stripClasses, '', $data['introtext']);
      } else {
        $data['introtext'] = '';
      }
      if (isset($data['fulltext']) && !empty($data['fullotext'])) {
        $data['fulltext'] = strip_tags($data['fulltext'], $allowedTags);
        $data['fulltext'] = preg_replace($stripClasses, '', $data['fulltext']);
      } else {
        $data['fulltext'] = '';
      }
  
      if ($data === false) {
      } else {
        yield (json_encode(
          $data
        ));
      }
    }
  } finally {
    // echo 'Done processing data' . PHP_EOL;
  }
};

$endPoint = function (string $givenBaseUrl, string $givenBasePath, int $givenResourceId = 0): string {
  return $givenResourceId ? sprintf('%s/%s/%s/%d', $givenBaseUrl, $givenBasePath, 'content/articles', $givenResourceId)
: sprintf('%s/%s/%s', $givenBaseUrl, $givenBasePath, 'content/articles');
};

$process = function (string $givenHttpVerb, string $endPoint, string   $dataString, array $headers, int $timeOut, $transport) {
  curl_setopt_array($transport, [
      CURLOPT_URL            => $endPoint,
      CURLOPT_RETURNTRANSFER => true,
      CURLOPT_ENCODING       => 'utf-8',
      CURLOPT_MAXREDIRS      => 10,
      CURLOPT_TIMEOUT        => $timeOut,
      CURLOPT_FOLLOWLOCATION => true,
      CURLOPT_SSLVERSION     => CURL_SSLVERSION_TLSv1_2,
      CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_2TLS,
      CURLOPT_CUSTOMREQUEST  => $givenHttpVerb,
      CURLOPT_POSTFIELDS     => $dataString,
      CURLOPT_HTTPHEADER     => $headers,
    ]
  );

  $response = curl_exec($transport);

  if (empty($response)) {
    throw new RuntimeException( 'Empty output', 422 );
  }
  return $response;
};

$dataSourceHttpVerb = 'GET';
$dataSourceDataString = '';
$dataSourceHeaders = [
  'Accept: application/json',
  'Accept-Encoding: deflate, gzip, br',
  'Content-Type: application/json',
  'Connection: keep-alive',
];
$dataSourceTimeout = 120;
$dataSourceTransport = curl_init();

try {
  $dataSourceResponse = $process($dataSourceHttpVerb, $dataSourceUrl, $dataSourceDataString, $dataSourceHeaders, $dataSourceTimeout, $dataSourceTransport);
  $postData = $generator($dataSourceResponse);

  $storage = [];
  foreach ($postData as $dataString) {
    if (!is_string($dataString)) {
      continue;
    }
    $curl = curl_init();
    try {
      $decodedDataString = json_decode($dataString);
      if ($decodedDataString === false) {
        continue;
      }
  
      $headers = [
        'Accept: application/vnd.api+json',
        'Content-Type: application/json',
        'Content-length: ' . mb_strlen($dataString),
        sprintf('X-Joomla-Token: %s', trim($token)),
      ];
      $pk = (int) $decodedDataString->id;
      $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $curl);
  
      $decodedJsonOutput = json_decode($output);
  
      if (isset($decodedJsonOutput->errors)) {
        $storage[] = ['mightExists' => $decodedJsonOutput->errors[0]->code === 400, 'decodedDataString' => $decodedDataString];
        continue;
      }
  
    } catch (Throwable $e) {
      echo $e->getMessage() . PHP_EOL;
      continue;
    } finally {
      curl_close($curl);
    }
  }

  foreach ($storage as $item) {
    $storageCurl = curl_init();
    try {
      if ($item['mightExists']) {
        $pk = (int) $item['decodedDataString']->id;
        $item['decodedDataString']->alias = sprintf('%s-%s', $item['decodedDataString']->alias, bin2hex(random_bytes(4)));
        // No need to do another json_encode anymore
        $dataString = json_encode($item['decodedDataString']);
        // HTTP request headers
        $headers = [
          'Accept: application/vnd.api+json',
          'Content-Type: application/json',
          'Content-Length: ' . mb_strlen($dataString),
          sprintf('X-Joomla-Token: %s', trim($token)),
        ];
        $output = $process($pk ? 'PATCH' : 'POST', $endPoint($baseUrl, $basePath, 0), $dataString, $headers, $timeOut, $storageCurl);
      }
    } catch (Throwable $storageThrowable) {
      echo $storageThrowable->getMessage() . PHP_EOL;
      continue;
    } finally {
      curl_close($storageCurl);
    }
  }
} catch (Throwable $e) {
  echo $e->getMessage() . PHP_EOL;
} finally {
  curl_close($dataSourceTransport);
}
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.