-1

这是 API 提供给我的 JSON 对象之一的示例。其中有 100 个。

[{"id": "133248644",
"associations": {"deals": {"results": [{"id": "2762673039",
                                          "type": "line_item_to_deal"}]}},
  "properties": {
            "createdate": "2020-08-06T15:05:23.253Z",
            "description": null,
            "hs_lastmodifieddate": "2020-08-06T15:05:23.253Z",
            "hs_object_id": "133248644",
            "name": "test product",
            "price": "100"},
 "createdAt": "2020-08-06T15:05:23.253Z",
 "updatedAt": "2020-08-06T15:05:23.253Z",
 "archived": false}]

除了嵌套在“关联”下的 id 之外，我想创建一个 pandas 数据框，它有一个 id 列以及与之关联的所有属性。本质上，我想删除嵌套在属性下的属性和嵌套在关联下的 id（以及重命名）。我该怎么办？

这是我尝试解决问题的可重现示例：

import json
import pandas as pd

response = """[{"id": "133248644",
"associations": {"deals": {"results": [{"id": "2762673039",
                                          "type": "line_item_to_deal"}]}},
  "properties": {
            "createdate": "2020-08-06T15:05:23.253Z",
            "description": null,
            "hs_lastmodifieddate": "2020-08-06T15:05:23.253Z",
            "hs_object_id": "133248644",
            "name": "test product",
            "price": "100"},
 "createdAt": "2020-08-06T15:05:23.253Z",
 "updatedAt": "2020-08-06T15:05:23.253Z",
 "archived": false}, 
{"id": "133345685",
 "associations": {"deals": {"results": [{"id": "2762673038",
                                          "type": "line_item_to_deal"}]}},
 "properties": {
             "createdate": 
             "2020-08-06T18:29:06.773Z", 
             "description": null,
             "hs_lastmodifieddate": "2020-08-06T18:29:06.773Z",
             "hs_object_id": "133345685",
             "name": "TEST PRODUCT 2",
             "price": "2222"},
 "createdAt": "2020-08-06T18:29:06.773Z", 
 "updatedAt": "2020-08-06T18:29:06.773Z",
 "archived": false}]"""


data = json.loads(response)
data_flat = [dict(id=x["id"], **x["properties"]) for x in data]

这是一个更好的解决方案，但仍然不是很完美。

data_flat = [dict(lineid=x["id"],dealid=x["associations"]["deals"]["results"][0]["id"], **x["properties"]) for x in data]

最后，这非常有用，但仍然需要我以一种复杂的方式从关联列中提取 id。

normal_data = pd.normalize_data(data)
4

1 回答 1

1
  • 处理list嵌套dicts 的 s 是复杂的。没有用于提取数据的可读单行。
  • 阅读data_pandas.json_normalize
  • associations.deals.results是s 的一个，用于list将中的每个分隔到单独的行dictpandas.DataFrame.explodedictlist
  • 使用.json_normalizeonassociations.deals.results将dicts 转换为列。
  • pandas.DataFrame.join df归一化列。
    • id已存在于数据框中，因此idindict将获得正确的后缀，但type不需要后缀，因为它不存在于df.
  • 用于pandas.DataFrame.rename重命名任何所需的列。
import pandas as pd
import json

# convert response from a string to a list of dicts
data = json.loads(response)

# create a pandas dataframe
df = pd.json_normalize(data)

# associations.deals.results is a list of dicts, explode them
df = df.explode('associations.deals.results').reset_index(drop=True)

# normalize the dicts in associations.deals.results and join them back to df
df = df.join(pd.json_normalize(df['associations.deals.results']), rsuffix='.associations.deals.results').drop(columns=['associations.deals.results'])

# display(df)
          id                 createdAt                 updatedAt  archived     properties.createdate properties.description properties.hs_lastmodifieddate properties.hs_object_id properties.name properties.price id.associations.deals.results               type
0  133248644  2020-08-06T15:05:23.253Z  2020-08-06T15:05:23.253Z     False  2020-08-06T15:05:23.253Z                   None       2020-08-06T15:05:23.253Z               133248644    test product              100                    2762673039  line_item_to_deal
1  133345685  2020-08-06T18:29:06.773Z  2020-08-06T18:29:06.773Z     False  2020-08-06T18:29:06.773Z                   None       2020-08-06T18:29:06.773Z               133345685  TEST PRODUCT 2             2222                    2762673038  line_item_to_deal

回复

response = """[{"id": "133248644",
"associations": {"deals": {"results": [{"id": "2762673039",
                                          "type": "line_item_to_deal"}]}},
  "properties": {
            "createdate": "2020-08-06T15:05:23.253Z",
            "description": null,
            "hs_lastmodifieddate": "2020-08-06T15:05:23.253Z",
            "hs_object_id": "133248644",
            "name": "test product",
            "price": "100"},
 "createdAt": "2020-08-06T15:05:23.253Z",
 "updatedAt": "2020-08-06T15:05:23.253Z",
 "archived": false}, 
{"id": "133345685",
 "associations": {"deals": {"results": [{"id": "2762673038",
                                          "type": "line_item_to_deal"}]}},
 "properties": {
             "createdate": 
             "2020-08-06T18:29:06.773Z", 
             "description": null,
             "hs_lastmodifieddate": "2020-08-06T18:29:06.773Z",
             "hs_object_id": "133345685",
             "name": "TEST PRODUCT 2",
             "price": "2222"},
 "createdAt": "2020-08-06T18:29:06.773Z", 
 "updatedAt": "2020-08-06T18:29:06.773Z",
 "archived": false}]"""
于 2020-08-25T21:36:56.807 回答