python - 使用 python 读取 json 网页

Question

我正在尝试读取 json 网页并将其插入数据框中。

import urllib3
import json
import pandas as pd

url = 'https://covid.ourworldindata.org/data/owid-covid-data.json'
http = urllib3.PoolManager()
r = http.request('GET', url, headers={'Accept': 'application/json'})

data = json.loads(r.data.decode('utf-8'))
data_copy = data

df = pd.DataFrame.from_dict(data_copy, orient='index')
df.transpose()

我需要使用我预览的相同代码，但输出必须采用以下格式：

Answer 1

这是此特定数据的解决方案：

req = requests.get("https://covid.ourworldindata.org/data/owid-covid-data.json")
d = json.loads(req.content)
records = [{**{"iso_code": key}, **item} for key, val in d.items() for item in val]

pd.DataFrame.from_records(records)

结果是：

Answer 2

• 此实现产生所需的输出形式。
• 只有pandas是必需的。
• 直接读取urlwithpandas.read_json
• 用于pandas.DataFrame.stack将字典的所有列堆叠成一列。
• 从 pandas v1.0.0 开始，json_normalize位于顶级名称空间中。
  • 否则，使用from pandas.io.json import json_normalize并替换pd.json_normalize(df2[0])为json_normalize(df2[0])

import pandas as pd

# read data with pandas
df = pd.read_json('https://covid.ourworldindata.org/data/owid-covid-data.json', orient='index')

# stack all the dictionary columns
df2 = df.stack().reset_index()

# use json_normalize to expand dicts, drop and rename columns
df2 = df2.join(pd.json_normalize(df2[0])).drop(columns=['level_1', 0]).rename(columns={'level_0': 'iso_code'})

iso_code location        date  total_cases  new_cases  total_deaths  new_deaths  total_cases_per_million  new_cases_per_million  total_deaths_per_million  new_deaths_per_million  stringency_index  population  population_density  median_age  aged_65_older  aged_70_older  gdp_per_capita  diabetes_prevalence  cvd_death_rate  handwashing_facilities  hospital_beds_per_100k  extreme_poverty  female_smokers  male_smokers  total_tests  total_tests_per_thousand tests_units  new_tests  new_tests_per_thousand  new_tests_smoothed  new_tests_smoothed_per_thousand
     ABW    Aruba  2020-03-13            2          2             0           0                   18.733                 18.733                       0.0                     0.0              0.00    106766.0               584.8        41.2         13.085          7.452       35973.781                11.62             NaN                     NaN                     NaN              NaN             NaN           NaN          NaN                       NaN         NaN        NaN                     NaN                 NaN                              NaN
     ABW    Aruba  2020-03-20            4          2             0           0                   37.465                 18.733                       0.0                     0.0             30.56    106766.0               584.8        41.2         13.085          7.452       35973.781                11.62             NaN                     NaN                     NaN              NaN             NaN           NaN          NaN                       NaN         NaN        NaN                     NaN                 NaN                              NaN
     ABW    Aruba  2020-03-24           12          8             0           0                  112.395                 74.930                       0.0                     0.0             41.67    106766.0               584.8        41.2         13.085          7.452       35973.781                11.62             NaN                     NaN                     NaN              NaN             NaN           NaN          NaN                       NaN         NaN        NaN                     NaN                 NaN                              NaN
     ABW    Aruba  2020-03-25           17          5             0           0                  159.227                 46.831                       0.0                     0.0             41.67    106766.0               584.8        41.2         13.085          7.452       35973.781                11.62             NaN                     NaN                     NaN              NaN             NaN           NaN          NaN                       NaN         NaN        NaN                     NaN                 NaN                              NaN
     ABW    Aruba  2020-03-26           19          2             0           0                  177.959                 18.733                       0.0                     0.0             41.67    106766.0               584.8        41.2         13.085          7.452       35973.781                11.62             NaN                     NaN                     NaN              NaN             NaN           NaN          NaN                       NaN         NaN        NaN                     NaN                 NaN                              NaN

Answer 3

而不是使用

pd.DataFrame.from_dictionary() 

利用

pd.DataFrame.from_records()

此方法将结构化或记录 ndarray 转换为 DataFrame。如果我有一些代码，我会尝试编写解决方案。