U.S. special data

This group contains categories that are specific to the United States. The market-relevant dataset in the U.S. is broader, timelier, and - for the most part - of better quality than on average in other countries. Also, U.S. data have on average greater global importance than those of other countries due to the dominant role of the USD for global financial markets. Unlike other category groups, the U.S. special dataset naturally does not come in panels.

Note that other internationally comparable U.S. data are grouped outside this data set under the respective international category, such as unemployment rates, external balances, or credit growth.

Business surveys

Ticker : ISMMANU_SA_3MMA / ISMMANU_SA_D3M3ML3

Label : ISM manufacturing survey, sa: 3mma / diff 3m/3m

Definition : ISM manufacturing survey headline index, seasonally adjusted: 3 month moving average / difference of latest 3 months over previous 3 months

Notes :

• The headline survey index is also known as manufacturing PMI.

• Data are released by the Institute for Supply Management (ISM).

• The survey uses responses executives from more than 400 industrial companies.

• The headline index is a composite based on five indicators: new orders, production, supplier deliveries, inventories, and employment.

Ticker : ISMSERV_SA_3MMA / ISMSERV_SA_D3M3ML3

Label : ISM services survey, sa: 3mma / diff 3m/3m

Definition : ISM services survey headline index, seasonally adjusted: 3 month moving average / difference of latest 3 months over previous 3 months

Notes :

• Data are relased by the Institute for Supply Management (ISM).

• The index is a composite based on four indicators: business activity, new orders, employment and supplier deliveries with equal weights.

Ticker : PHILMANU_SA_3MMA / PHILMANU_SA_D3M3ML3

Label : Philadelphia manufacturing survey, sa: 3mma / diff 3m/3m

Definition : Federal Reserve Bank of Philadelphia manufacturing survey, general activity, seasonally adjusted: 3 month moving average / difference of latest 3 months over previous 3 months

Notes :

• The Philadelphia Fed “Manufacturing Business Outlook Survey” is a monthly survey of manufacturers in the Third Federal Reserve District (Delaware, New Jersey, and Pennsylvania), with a history back to 1968. The index is based on the following sub-categories: employment, working hours, new and unfilled orders, shipments, inventories, delivery times, prices paid, and prices received.

Ticker : NYMANU_SA_3MMA / NYMANU_SA_D3M3ML3

Label : Empire State manufacturing survey, sa: 3mma / diff 3m/3m

Definition : Empire State manufacturing survey, general business conditions, seasonally adjusted: 3 month moving average / difference of latest 3 months over previous 3 months

Notes :

• This is a monthly survey conducted by the Federal Reserve Bank of New York, covering about 200 manufactoring executives in New York State.

Composite activity indicator

Ticker : CFNAI_SA_3MMA / CFNAI_SA_D3M3ML3

Label : Chicago Fed National Activity Index, sa: 3mma / diff 3m/3m

Definition : Chicago Fed National Activity Index, seasonally adjusted: 3 month moving average / difference of latest 3 months over previous 3 months

Notes :

• “The Chicago Fed National Activity Index (CFNAI) is a weighted average of 85 monthly indicators of national economic activity. The CFNAI provides a single summary measure of a common factor in these national economic data. As such, historical movements in this Chicago Fed index closely track periods of economic expansion and contraction, as well as periods of increasing and decreasing inflationary pressure.” Chicago Federal Reserve

• “The index is a weighted average of 85 existing monthly indicators of growth in national economic activity drawn from four broad categories of data: (1) production and income; (2) employment, unemployment, and hours; (3) personal consumption and housing; and (4) sales, orders, and inventories… Put simply, the index is the first principal component of the 85 series. If all 85 series were proportional to a single common variable plus individual noise discrepancies, the CFNAI would be the estimate of the common variable that minimizes the implied noise discrepancies in a least-squares sense… A zero value for the monthly index has been associated with the national economy expanding at its historical trend (average) rate of growth; negative values with below-average growth (in standard deviation units); and positive values with above-average growth… Month-to-month movements can be volatile, so the monthly index’s three-month moving average… provides a more consistent picture of national economic growth.” Chicago Federal Reserve

• Although CFNAI data series go back to the 1970s, the index has only been released since 2001. Hence the information states of this quantamental indicator only begin in 2001. Given that the index value relies heavily on estimation, strategy backetests for older observation periods would be invalid and misleading.

• The index is considered seasonally adjusted by virtue of all constituent series being seasonally adjusted.

Orders and inventories

Ticker : DGORDERS_SA_P1M1ML1 / DGORDERS_SA_P3M3ML3AR / DGORDERS_SA_P1M1ML12 / DGORDERS_SA_P1M1ML12_3MMA

Label : Durable goods orders, sa: %m/m / % 3m/3m AR / %oya / %oya, 3mma

Definition : New durable goods orders: % month-on-month / % 3-month-moving average over prevous 3-month moving average annualized rate / % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data were obtained through sampling information of the U.S. Census Bureau, from manufacturers with $500 million annual shipments.

• Information on what constitutes a durable good can be found here .

Ticker : DGORDERSXD_SA_P1M1ML1 / DGORDERSXD_SA_P3M3ML3AR / DGORDERSXD_SA_P1M1ML12 / DGORDERSXD_SA_P1M1ML12_3MMA

Label : Durable goods orders ex defense, sa: %m/m / % 3m/3m AR / %oya / %oya, 3mma

Definition : New durable goods orders excluding defense: % month-on-month / % 3-month-moving average over previous 3-month moving average annualized rate / % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data were obtained through sampling information of the U.S. Census Bureau, from manufacturers with $500 million annual shipments, excluding the defense industry.

• Infomation on what constitutes a durable good can be found here .

Ticker : BINVENTORIES_SA_P1M1ML1 / BINVENTORIES_SA_P3M3ML3AR / BINVENTORIES_SA_P1M1ML12 / BINVENTORIES_SA_P1M1ML12_3MMA

Label : Business inventories, sa: %m/m / % 3m/3m AR / %oya / %oya, 3mma

Definition : Business inventories, seasonally adjusted: % month-on-month / % 3-month-moving average over previous 3-month moving average annualized rate / % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data were obtained through sampling information of the U.S. Census Bureau, and measure the dollar value of inventories of the U.S. domestic manufacturing and trade businesses.

Private consumption

Ticker : CBCCONF_SA_3MMA / CBCCONF_SA_D3M3ML3

Label : Conference board consumer confidence, sa: 3mma / diff 3m/3m

Definition : Conference board consumer confidence index, seasonally adjusted: 3 month moving average / difference of latest 3 months over previous 3 months

Notes :

• The survey refers to current conditions and likely developments for the months ahead. It is sampling consumer attitudes, buying intentions, vacation plans and consumer expectations for inflation, stock prices and interest rates. Data are data available by age, income, region and top 8 states.

• The confidence index is “a barometer of the health of the U.S. economy from the perspective of the consumer. The index is based on consumers’ perceptions of current business and employment conditions, as well as their expectations for the following six months regarding business conditions, employment, and income. The Consumer Confidence Index and its related series are among the earliest sets of economic indicators available each month and are closely watched as leading indicators for the U.S. economy.” See these technical notes for further information.

Ticker : UMCCONF_NSA_3MMA / UMCCONF_NSA_D3M3ML3

Label : University of Michigan consumer sentiment, sa: 3mma / %oya, 3mma /diff 3m/3m

Definition : University of Michigan consumer sentiment index, seasonally adjusted: 3 month moving average / difference of latest 3 months over previous 3 months

Notes :

• The University of Michigan Consumer Sentiment Index is a consumer confidence index published monthly by the University of Michigan. The index is normalized to have a value of 100 in the first quarter of 1966. Each month, at least 500 telephone interviews are conducted as part of a contiguous United States sample. See Wikipedia for more information.

• Although the survey is conducted by the University of Michigan it polls members of the public from 48 States and the District of Columbia. Fifty core questions are asked. See here for more details.

Ticker : CONS_SA_P1M1ML12 / CONS_SA_P1M1ML12_3MMA / CONS_SA_P3M3ML3AR

Label : Real consumption, sa: %oya / %oya, 3mma / %3m/3m AR

Definition : Real personal consumption, seasonally adjusted: % over a year ago / % over a year ago, 3-month moving average / % 3 months over previous 3 months annualized rate

Notes :

• Consumption data are from the U.S. Bureau of Economic Analysis (BEA) and measure the volume of goods and services purchased by households in the United States.

Ticker : RSALES_SA_P1M1ML1 / RSALES_SA_P6M6ML6AR / RSALES_SA_P1M1ML12 / RSALES_SA_P1M1ML12_3MMA / RSALESXAF_SA_P1M1ML1 / RSALESXAF_SA_P6M6ML6AR / RSALESXAF_SA_P1M1ML12 / RSALESXAF_SA_P1M1ML12_3MMA

Label : Retail sales: total, sa, %m/m / total, sa, %6m/6m AR / total, sa, %oya / total, sa, %oya, 3mma / ex autos and gas, sa, %m/m / ex autos and gas, sa, %6m/6m AR / ex autos and gas, sa, %oya / ex autos and gas, sa, %oya, 3mma

Definition : Retail sales values: total, sa, %m/m / total, sa, %6m/6m annualized rate / total, nsa, %oya / total, nsa, %oya 3mma / ex autos and gas, sa, %m/m / ex autos and gas, sa, %6m/6m annualized rate / ex autos and gas, nsa, %oya / ex autos and gas, nsa, %oya 3mma

Notes :

• Retail sales data come from the U.S. Census Bureau and are in U.S. dollar, not volumes.

• Data covers retail and food service companies with one or more establishments that sell merchandise and associated services to final consumers.

• The “ex autos and gas” series excludes motor vehicles and parts as well as sales from gasoline stations

Ticker : PERSINC_SA_P1M1ML12 / PERSINC_SA_P1M1ML12_3MMA / PERSINC_SA_P3M3ML3AR

Label : Personal income, sa: %oya / %oya, 3mma / %3m/3m AR

Definition : Personal income, seasonally adjusted: % over a year ago / % over a year ago, 3-month moving average / % 3 months over previous 3 months annualized rate

Notes :

• Personal income data are from the U.S. Bureau of Economic Analysis (BEA). The data include all income sources, including wages and salaries, government benefits, dividends and interest, and business ownership.

Ticker : CARSALES_SA_P1M1ML12 / CARSALES_SA_P1M1ML12_3MMA

Label : Moto vehicle sales, sa: %oya / %oya, 3mma

Definition : Moto vehicle sales, units, seasonally adjusted: % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data are from the U.S. Bureau of Economic Analysis (BEA).

• These comprise the sales of light trucks up to 14,000 pounds and all passenger cars, including station wagons and minivans.

Housing indicators

Ticker : CONSTRSPEND_SA_P1M1ML12 / CONSTRSPEND_SA_P1M1ML12_3MMA

Label : Construction spending, sa: %oya / %oya, 3mma

Definition : Construction spending in USD: % over a year ago / % over a year ago, 3-month moving average

Notes :

• These are estimates from the U.S. Census Bureau on the total dollar value of construction spending, including new constructions and improvements to existing structures, both public and private spending.

Ticker : NAHBSURV_SA_3MMA / NAHBSURV_SA_D3M3ML3

Label : Homebuilders survey, sa: 3mma / diff 3m/3m

Definition : Housing market index of the homebuilders survey, seasonally adjusted: 3 month moving average / difference of latest 3 months over previous 3 months

Notes :

• This is the Housing Market Index of the National Association of Homebuilders (NAHB) and Wells Fargo. It is based on a monthly survey of NAHB members regarding the single-family housing market. The survey asks respondents to rate market conditions for the sale of new homes at the present time and in the next six months as well as the traffic of prospective buyers of new homes.

Ticker : NEWHOMESALES_SA_P1M1ML12 / NEWHOMESALES_SA_P1M1ML12_3MMA / EXISTHOMESALES_SA_P1M1ML12 / EXISTHOMESALES_SA_P1M1ML12_3MMA

Label : Home sales: %oya, new / %oya, 3mma, new / %oya, existing / %oya, 3mma, existing

Definition : Home sales: % over a year ago, new / % over a year ago, 3-month average, new / % over a year ago, existing / % over a year ago, 3-month average, existing

Notes :

• The data are from the U.S. Census Bureau and essentially inform on the number of new and existing single-family houses sold.

Ticker : PENDHOMESALES_SA_P1M1ML12 / PENDHOMESALES_SA_P1M1ML12_3MMA

Label : Pending homes sales: %oya / %oya, 3mma

Definition : Pending home sales: percentage change over a year ago / percentage change over a year ago, 3-month average, change over the last three estimable months.

Notes :

• The data are from the National Association of Realtors (NAR) and record homes under contract to be sold and not sold yet.

• The pending home sales index includes contracts for existing single-family homes, condos, and co-ops. Because a home goes under contract a month or two before it is sold, the index generally leads existing home sales by a month or two.

Labor market

Ticker : INJLCLAIMS_SA_D4W4WL4 / INJLCLAIMS_SA_D1W1WL13 / INJLCLAIMS_SA_D13W13WL13 / INJLCLAIMS_SA_D4W4WL52 / INJLCLAIMS_SA_4WMAv5YMM / INJLCLAIMS_SA_4WMAv10YMM

Label : Initial jobless claims, sa: dif 4w/4w / dif oqa / dif oqa, 13wma / dif oya, 4wma / %4wma vs 5yma / %4wma vs 10yma

Definition : Initial jobless claims, seasonally adjusted: difference 4-week moving average over previous 4-week moving average / difference over a quarter ago / difference over a quarter ago, 13-week moving average / difference over a year ago, 4-week moving average / % 4-week moving average over previous 5-year moving average / % 4-week moving average over previous 10-year moving average

Notes :

• The data are from the U.S. Department of Labor and based on the number of people filing for unemployment insurance. It is one of the earliest indicators of the labor market situation in the U.S.

Ticker : COJLCLAIMS_SA_D4W4WL4 / COJLCLAIMS_SA_D1W1WL13 / COJLCLAIMS_SA_D13W13WL13 / COJLCLAIMS_SA_D4W4WL52 / COJLCLAIMS_SA_4WMAv5YMM / COJLCLAIMS_SA_4WMAv10YMM

Label : Continuing jobless claims, sa: dif 4w/4w / dif oqa / dif oqa, 13wma / dif oya, 4wma / %4wma vs 5yma / %4wma vs 10yma

Definition : Continuing jobless claims, seasonally adjusted: difference 4-week moving average over previous 4-week moving average / difference over a quarter ago / difference over a quarter ago, 13-week moving average / difference over a year ago, 4-week moving average / % 4-week moving average over previous 5-year moving average / % 4-week moving average over previous 10-year moving average

Notes :

• The data are from the U.S. Department of Labor and based on the number of people filing for unemployment insurance. It is one of the timeliest indicators of the labor market situation in the U.S.

Ticker : NFPAYROLLPRIV_SA_P1M1ML1 / NFPAYROLLPRIV_SA_P3M3ML3AR / NFPAYROLLPRIV_SA_P1M1ML12 / NFPAYROLLPRIV_SA_P1M1ML12_3MMA

Label : Non-farm payroll, private sector, sa: %m/m / %3m/3m / %oya / %oya, 3mma

Definition : Non-farm payroll, private sector, seasonally adjusted: % month-on-month / % 3-month moving average over previous 3-month moving average / % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data are from the U.S. Bureau of Labor Statistics (BLS) and based on the number of people paid for employment excluding farm-workers.

Ticker : ADPEMPL_SA_P1M1ML1 / ADPEMPL_SA_P3M3ML3AR / ADPEMPL_SA_P1M1ML12 / ADPEMPL_SA_P1M1ML12_3MMA

Label : ADP employment, sa: %m/m / %3m/3m / %oya / %oya, 3mma

Definition : ADP employment, seasonally adjusted: % month-on-month / % 3-month moving average over previous 3-month moving average / % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data are from Automatic Data Processing, Inc. (ADP) and based on the number of people in private employment.

Ticker : HOURLYWAGES_SA_P1M1ML1 / HOURLYWAGES_SA_P3M3ML3AR / HOURLYWAGES_SA_P1M1ML12 / HOURLYWAGES_SA_P1M1ML12_3MMA

Label : Hourly wages, sa: %m/m / %3m/3m AR / %oya / %oya, 3mma

Definition : Average hourly wages, seasonally adjusted: % month-on-month / % 3-month moving average over previous 3-month moving average annualized rate / % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data comprises average hourly earnings of all employees, collected from the U.S. Bureau of Labor Statistics (BLS).

Ticker : HOURLYCOMP_SA_P1Q1QL4

Label : Hourly compensation, sa: % oya

Definition : Hourly compensation, seasonally adjusted: % over a year ago

Notes :

• Labor compensation includes monetary and nonmonetary payments to or on behalf of individuals for labor services used to produce output. Workers may be employees, proprietors, and self-employed or unpaid family workers. Compensation of employees includes salaries, wages, bonuses, contributions to benefits plans, and other forms of payment. The compensation of employees in general government, nonprofit institutions, and private households are subtracted from compensation of employees in domestic industries to derive employee compensation for the business sector. For the nonfarm business sector, compensation further excludes farm employees. Proprietor’s income is computed independently and scaled to control totals as described above.” See here for more information.

• Data are from the U.S. Bureau of Labor Statistics (BLS) and apply to the nonfarm business sector.

Ticker : HOURLYPROD_SA_P1Q1QL4

Label : Labor productivity, sa: % oya

Definition : Labor productivity (output per hour), seasonally adjusted: % over a year ago

Notes :

• Labor productivity, or output per hour, is calculated by dividing an index of real output by an index of hours worked by all persons, including employees, proprietors, and unpaid family workers in the nonfarm business sector.

• Data are from the U.S. Bureau of Labor Statistics (BLS).

Prices

Ticker : CPIUH_SA_P1M1ML1 / CPIUH_SA_P6M6ML6AR / CPIUH_NSA_P1M1ML12 / CPIUH_NSA_P1M1ML12_3MMA

Label : Consumer price index, sa: %m/m / %6m/6m AR / %oya / %oya, 3mma

Definition : Headline consumer price index for urban areas, seasonally adjusted: % month-on-month / % 6-month moving average over previous 6-month moving average annualized rate / % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data are from the U.S. Bureau of Labor Statistics (BLS).

Ticker : CPIUC_SA_P1M1ML1 / CPIUC_SA_P6M6ML6AR / CPIUC_NSA_P1M1ML12 / CPIUC_NSA_P1M1ML12_3MMA

Label : Core CPI, sa: %m/m / %6m/6m AR / %oya / %oya, 3mma

Definition : Core consumer price index for urban areas, seasonally adjusted: % month-on-month / % 6-month moving average over previous 6-month moving average annualized rate / % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data are from the U.S. Bureau of Labor Statistics (BLS) and the CPI excludes food and energy.

Ticker : PPIH_SA_P1M1ML1

Label : Producer price index, sa: %m/m

Definition : Producer price index, seasonally adjusted: % month-on-month

Notes :

• The producer price index measures average changes in prices received by domestic producers for their output. It includes mining, forestry, utility, construction, manufacturing, and services industries.

• Data are collected by the U.S. Bureau of Labor Statistics (BLS).

Ticker : PPIC_SA_P1M1ML1 / PPIC_SA_P6M6ML6AR / PPIC_NSA_P1M1ML12 / PPIC_NSA_P1M1ML12_3MMA

Label : Core PPI, sa: %m/m / %6m/6m AR / %oya / %oya, 3mma

Definition : Core PPI, seasonally adjusted: % month-on-month / % 6-month moving average over previous 6-month moving average annualized rate / % over a year ago / % over a year ago, 3-month moving average

Notes :

• The producer price index measures average changes in prices received by domestic producers for their output. It includes mining, forestry, utility, construction, manufacturing, and services industries.

• Data are collected by the U.S. Bureau of Labor Statistics (BLS) and exclude food and energy.

Ticker : IMPIH_NSA_P1M1ML12 / IMPIH_NSA_P1M1ML12_3MMA

Label : Import price index: %oya / %oya, 3mma

Definition : Import price index: % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data are from the U.S. Bureau of Labor Statistics (BLS) and they exclude military goods.

Ticker : IMPIC_NSA_P1M1ML12 / IMPIC_NSA_P1M1ML12_3MMA

Label : Core import price index: %oya / %oya, 3mma

Definition : Import price index excluding fuel: % over a year ago / % over a year ago, 3-month moving average

Notes :

• Data are from the U.S. Bureau of Labor Statistics (BLS) and they exclude military goods and fuel.

Imports

Only the standard Python data science packages and the specialized macrosynergy package are needed.

import os
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import math

import json
import yaml

import macrosynergy.management as msm
import macrosynergy.panel as msp
import macrosynergy.signal as mss
import macrosynergy.pnl as msn


from macrosynergy.download import JPMaQSDownload

from timeit import default_timer as timer
from datetime import timedelta, date, datetime

import warnings

warnings.simplefilter("ignore")

The JPMaQS indicators we consider are downloaded using the J.P. Morgan Dataquery API interface within the macrosynergy package. This is done by specifying ticker strings , formed by appending an indicator category code <category> to a currency area code <cross_section> . These constitute the main part of a full quantamental indicator ticker, taking the form DB(JPMAQS,<cross_section>_<category>,<info>) , where <info> denotes the time series of information for the given cross-section and category. The following types of information are available:

• value giving the latest available values for the indicator

• eop_lag referring to days elapsed since the end of the observation period

• mop_lag referring to the number of days elapsed since the mean observation period

• grade denoting a grade of the observation, giving a metric of real time information quality.

After instantiating the JPMaQSDownload class within the macrosynergy.download module, one can use the download(tickers,start_date,metrics) method to easily download the necessary data, where tickers is an array of ticker strings, start_date is the first collection date to be considered and metrics is an array comprising the times series information to be downloaded.

cids = ["USD"]

main = [
    "NYMANU_SA_3MMA",
    "NYMANU_SA_D3M3ML3",
    "PHILMANU_SA_3MMA",
    "PHILMANU_SA_D3M3ML3",
    "CBCCONF_SA_3MMA",
    "CBCCONF_SA_D3M3ML3",
    "NAHBSURV_SA_3MMA",
    "NAHBSURV_SA_D3M3ML3",
    "ISMMANU_SA_3MMA",
    "ISMMANU_SA_D3M3ML3",
    "ISMSERV_SA_3MMA",
    "ISMSERV_SA_D3M3ML3",
    "UMCCONF_NSA_3MMA",
    "UMCCONF_NSA_D3M3ML3",
    "CONSTRSPEND_SA_P1M1ML12_3MMA",
    "CONSTRSPEND_SA_P1M1ML12",
    "EXISTHOMESALES_SA_P1M1ML12_3MMA",
    "EXISTHOMESALES_SA_P1M1ML12",
    "NEWHOMESALES_SA_P1M1ML12_3MMA",
    "NEWHOMESALES_SA_P1M1ML12",
    "PENDHOMESALES_SA_P1M1ML12_3MMA",
    "PENDHOMESALES_SA_P1M1ML12",
    "BINVENTORIES_SA_P1M1ML12",
    "BINVENTORIES_SA_P1M1ML1",
    "BINVENTORIES_SA_P3M3ML3AR",
    "BINVENTORIES_SA_P1M1ML12_3MMA",
    "DGORDERS_SA_P1M1ML12",
    "DGORDERS_SA_P1M1ML1",
    "DGORDERS_SA_P3M3ML3AR",
    "DGORDERS_SA_P1M1ML12_3MMA",
    "DGORDERSXD_SA_P1M1ML12",
    "DGORDERSXD_SA_P1M1ML1",
    "DGORDERSXD_SA_P3M3ML3AR",
    "DGORDERSXD_SA_P1M1ML12_3MMA",
    "NFPAYROLLPRIV_SA_P1M1ML12",
    "NFPAYROLLPRIV_SA_P1M1ML1",
    "NFPAYROLLPRIV_SA_P3M3ML3AR",
    "NFPAYROLLPRIV_SA_P1M1ML12_3MMA",
    "HOURLYWAGES_SA_P1M1ML12",
    "HOURLYWAGES_SA_P1M1ML1",
    "HOURLYWAGES_SA_P3M3ML3AR",
    "HOURLYWAGES_SA_P1M1ML12_3MMA",
    "ADPEMPL_SA_P1M1ML12",
    "ADPEMPL_SA_P1M1ML1",
    "ADPEMPL_SA_P3M3ML3AR",
    "ADPEMPL_SA_P1M1ML12_3MMA",
    "INJLCLAIMS_SA_D4W4WL4",
    "INJLCLAIMS_SA_D4W4WL52",
    "INJLCLAIMS_SA_D1W1WL13",
    "INJLCLAIMS_SA_D13W13WL13",
    "COJLCLAIMS_SA_D4W4WL4",
    "COJLCLAIMS_SA_D4W4WL52",
    "COJLCLAIMS_SA_D1W1WL13",
    "COJLCLAIMS_SA_D13W13WL13",
    "INJLCLAIMS_SA_4WMAv5YMM",
    "INJLCLAIMS_SA_4WMAv10YMM",
    "COJLCLAIMS_SA_4WMAv5YMM",
    "COJLCLAIMS_SA_4WMAv10YMM",
    "IMPIH_NSA_P1M1ML12",
    "IMPIH_NSA_P1M1ML12_3MMA",
    "IMPIC_NSA_P1M1ML12",
    "IMPIC_NSA_P1M1ML12_3MMA",
    "RSALES_SA_P1M1ML1",
    "RSALES_SA_P6M6ML6AR",
    "RSALES_SA_P1M1ML12_3MMA",
    "RSALES_SA_P1M1ML12",
    "RSALESXAF_SA_P1M1ML1",
    "RSALESXAF_SA_P6M6ML6AR",
    "RSALESXAF_SA_P1M1ML12_3MMA",
    "RSALESXAF_SA_P1M1ML12",
    "CONS_SA_P1M1ML12",
    "CONS_SA_P3M3ML3AR",
    "CONS_SA_P1M1ML12_3MMA",
    "PERSINC_SA_P1M1ML12",
    "PERSINC_SA_P3M3ML3AR",
    "PERSINC_SA_P1M1ML12_3MMA",
    "HOURLYPROD_SA_P1Q1QL4",
    "HOURLYCOMP_SA_P1Q1QL4",
    "CARSALES_SA_P1M1ML12_3MMA",
    "CARSALES_SA_P1M1ML12",
    "CFNAI_SA_3MMA",
    "CFNAI_SA_D3M3ML3"
]

cpi = [
    "CPIUH_SA_P1M1ML1",
    "CPIUH_SA_P6M6ML6AR",
    "CPIUH_NSA_P1M1ML12",
    "CPIUH_NSA_P1M1ML12_3MMA",
    "CPIUC_SA_P1M1ML1",
    "CPIUC_SA_P6M6ML6AR",
    "CPIUC_NSA_P1M1ML12",
    "CPIUC_NSA_P1M1ML12_3MMA",
    "PPIH_SA_P1M1ML1",
    "PPIH_SA_P6M6ML6AR",
    "PPIH_NSA_P1M1ML12",
    "PPIH_NSA_P1M1ML12_3MMA",
    "PPIC_SA_P1M1ML1",
    "PPIC_SA_P6M6ML6AR",
    "PPIC_NSA_P1M1ML12",
    "PPIC_NSA_P1M1ML12_3MMA",
]

econ = [
    "GBF02YXR_NSA",
    "GBF05YXR_NSA",
    "GBF10YXR_NSA",
    "EQXR_NSA",
    "DU05YXR_NSA"
    
]
xcats = main + cpi

# Download series from J.P. Morgan DataQuery by tickers

start_date = "1990-01-01"
tickers = [cid + "_" + xcat for cid in cids for xcat in (xcats+econ)]
print(f"Maximum number of tickers is {len(tickers)}")

# Retrieve credentials

client_id: str = os.getenv("DQ_CLIENT_ID")
client_secret: str = os.getenv("DQ_CLIENT_SECRET")

# Download from DataQuery

with JPMaQSDownload(client_id=client_id, client_secret=client_secret) as downloader:
    start = timer()
    assert downloader.check_connection()
    df = downloader.download(
        tickers=tickers,
        start_date=start_date,
        show_progress=True,
        metrics=["value", "eop_lag", "mop_lag", "grading"],
        suppress_warning=True,
    )
    end = timer()

dfd = df

print("Download time from DQ: " + str(timedelta(seconds=end - start)))

Maximum number of tickers is 103
Downloading data from JPMaQS.
Timestamp UTC:  2024-11-07 10:24:31
Connection successful!

Requesting data: 100%|██████████| 21/21 [00:04<00:00,  4.83it/s]
Downloading data: 100%|██████████| 21/21 [00:13<00:00,  1.59it/s]

Some dates are missing from the downloaded data. 
3 out of 9096 dates are missing.
Download time from DQ: 0:00:21.100434

# Assign categories to high level labels to make subsequent analysis more readable

cat_dict = {
    "Business": ["NYMANU", "PHILMANU", "ISMSERV", "ISMMANU","CFNAI"],
    "Orders": ["BINVENTORIES", "DGORDERS", "DGORDERSXD"],
    "Consumption": [
        "CBCCONF",
        "UMCCONF",
        "CARSALES",
        "RSALES",
        "CONS",
        "PERSINC",
        "RSALESXAF",
    ],
    "Housing": [
        "NAHBSURV",
        "EXISTHOMESALES",
        "NEWHOMESALES",
        "PENDHOMESALES",
        "CONSTRSPEND",
    ],
    "Labor": [
        "INJLCLAIMS",
        "COJLCLAIMS",
        "ADPEMPL",
        "NFPAYROLLPRIV",
        "HOURLYWAGES",
        "HOURLYCOMP",
        "HOURLYPROD",
    ],
    "Prices": ["CPIUH", "CPIUC", "IMPIC", "IMPIH", "PPIH", "PPIC"],
}
order = ["Business", "Orders", "Consumption", "Housing", "Labor", "Prices"]


def cat_dict_to_column(cat_string, cat_dict):
    for cat_list in cat_dict:
        if cat_string in cat_dict[cat_list]:
            cat_keys = [*cat_dict.keys()]

            return cat_keys.index(cat_list)
        else:
            continue


def cat_dict_to_key(cat_string, cat_dict):
    for cat_list in cat_dict:
        if cat_string in cat_dict[cat_list]:
            return cat_list
        else:
            continue

Availability

cids_exp = cids

msm.missing_in_df(dfd, xcats=xcats, cids=["USD"])

Show code cell output Hide code cell output

No missing XCATs across DataFrame.
Missing cids for ADPEMPL_SA_P1M1ML1:               []
Missing cids for ADPEMPL_SA_P1M1ML12:              []
Missing cids for ADPEMPL_SA_P1M1ML12_3MMA:         []
Missing cids for ADPEMPL_SA_P3M3ML3AR:             []
Missing cids for BINVENTORIES_SA_P1M1ML1:          []
Missing cids for BINVENTORIES_SA_P1M1ML12:         []
Missing cids for BINVENTORIES_SA_P1M1ML12_3MMA:    []
Missing cids for BINVENTORIES_SA_P3M3ML3AR:        []
Missing cids for CARSALES_SA_P1M1ML12:             []
Missing cids for CARSALES_SA_P1M1ML12_3MMA:        []
Missing cids for CBCCONF_SA_3MMA:                  []
Missing cids for CBCCONF_SA_D3M3ML3:               []
Missing cids for CFNAI_SA_3MMA:                    []
Missing cids for CFNAI_SA_D3M3ML3:                 []
Missing cids for COJLCLAIMS_SA_4WMAv10YMM:         []
Missing cids for COJLCLAIMS_SA_4WMAv5YMM:          []
Missing cids for COJLCLAIMS_SA_D13W13WL13:         []
Missing cids for COJLCLAIMS_SA_D1W1WL13:           []
Missing cids for COJLCLAIMS_SA_D4W4WL4:            []
Missing cids for COJLCLAIMS_SA_D4W4WL52:           []
Missing cids for CONSTRSPEND_SA_P1M1ML12:          []
Missing cids for CONSTRSPEND_SA_P1M1ML12_3MMA:     []
Missing cids for CONS_SA_P1M1ML12:                 []
Missing cids for CONS_SA_P1M1ML12_3MMA:            []
Missing cids for CONS_SA_P3M3ML3AR:                []
Missing cids for CPIUC_NSA_P1M1ML12:               []
Missing cids for CPIUC_NSA_P1M1ML12_3MMA:          []
Missing cids for CPIUC_SA_P1M1ML1:                 []
Missing cids for CPIUC_SA_P6M6ML6AR:               []
Missing cids for CPIUH_NSA_P1M1ML12:               []
Missing cids for CPIUH_NSA_P1M1ML12_3MMA:          []
Missing cids for CPIUH_SA_P1M1ML1:                 []
Missing cids for CPIUH_SA_P6M6ML6AR:               []
Missing cids for DGORDERSXD_SA_P1M1ML1:            []
Missing cids for DGORDERSXD_SA_P1M1ML12:           []
Missing cids for DGORDERSXD_SA_P1M1ML12_3MMA:      []
Missing cids for DGORDERSXD_SA_P3M3ML3AR:          []
Missing cids for DGORDERS_SA_P1M1ML1:              []
Missing cids for DGORDERS_SA_P1M1ML12:             []
Missing cids for DGORDERS_SA_P1M1ML12_3MMA:        []
Missing cids for DGORDERS_SA_P3M3ML3AR:            []
Missing cids for EXISTHOMESALES_SA_P1M1ML12:       []
Missing cids for EXISTHOMESALES_SA_P1M1ML12_3MMA:  []
Missing cids for HOURLYCOMP_SA_P1Q1QL4:            []
Missing cids for HOURLYPROD_SA_P1Q1QL4:            []
Missing cids for HOURLYWAGES_SA_P1M1ML1:           []
Missing cids for HOURLYWAGES_SA_P1M1ML12:          []
Missing cids for HOURLYWAGES_SA_P1M1ML12_3MMA:     []
Missing cids for HOURLYWAGES_SA_P3M3ML3AR:         []
Missing cids for IMPIC_NSA_P1M1ML12:               []
Missing cids for IMPIC_NSA_P1M1ML12_3MMA:          []
Missing cids for IMPIH_NSA_P1M1ML12:               []
Missing cids for IMPIH_NSA_P1M1ML12_3MMA:          []
Missing cids for INJLCLAIMS_SA_4WMAv10YMM:         []
Missing cids for INJLCLAIMS_SA_4WMAv5YMM:          []
Missing cids for INJLCLAIMS_SA_D13W13WL13:         []
Missing cids for INJLCLAIMS_SA_D1W1WL13:           []
Missing cids for INJLCLAIMS_SA_D4W4WL4:            []
Missing cids for INJLCLAIMS_SA_D4W4WL52:           []
Missing cids for ISMMANU_SA_3MMA:                  []
Missing cids for ISMMANU_SA_D3M3ML3:               []
Missing cids for ISMSERV_SA_3MMA:                  []
Missing cids for ISMSERV_SA_D3M3ML3:               []
Missing cids for NAHBSURV_SA_3MMA:                 []
Missing cids for NAHBSURV_SA_D3M3ML3:              []
Missing cids for NEWHOMESALES_SA_P1M1ML12:         []
Missing cids for NEWHOMESALES_SA_P1M1ML12_3MMA:    []
Missing cids for NFPAYROLLPRIV_SA_P1M1ML1:         []
Missing cids for NFPAYROLLPRIV_SA_P1M1ML12:        []
Missing cids for NFPAYROLLPRIV_SA_P1M1ML12_3MMA:   []
Missing cids for NFPAYROLLPRIV_SA_P3M3ML3AR:       []
Missing cids for NYMANU_SA_3MMA:                   []
Missing cids for NYMANU_SA_D3M3ML3:                []
Missing cids for PENDHOMESALES_SA_P1M1ML12:        []
Missing cids for PENDHOMESALES_SA_P1M1ML12_3MMA:   []
Missing cids for PERSINC_SA_P1M1ML12:              []
Missing cids for PERSINC_SA_P1M1ML12_3MMA:         []
Missing cids for PERSINC_SA_P3M3ML3AR:             []
Missing cids for PHILMANU_SA_3MMA:                 []
Missing cids for PHILMANU_SA_D3M3ML3:              []
Missing cids for PPIC_NSA_P1M1ML12:                []
Missing cids for PPIC_NSA_P1M1ML12_3MMA:           []
Missing cids for PPIC_SA_P1M1ML1:                  []
Missing cids for PPIC_SA_P6M6ML6AR:                []
Missing cids for PPIH_NSA_P1M1ML12:                []
Missing cids for PPIH_NSA_P1M1ML12_3MMA:           []
Missing cids for PPIH_SA_P1M1ML1:                  []
Missing cids for PPIH_SA_P6M6ML6AR:                []
Missing cids for RSALESXAF_SA_P1M1ML1:             []
Missing cids for RSALESXAF_SA_P1M1ML12:            []
Missing cids for RSALESXAF_SA_P1M1ML12_3MMA:       []
Missing cids for RSALESXAF_SA_P6M6ML6AR:           []
Missing cids for RSALES_SA_P1M1ML1:                []
Missing cids for RSALES_SA_P1M1ML12:               []
Missing cids for RSALES_SA_P1M1ML12_3MMA:          []
Missing cids for RSALES_SA_P6M6ML6AR:              []
Missing cids for UMCCONF_NSA_3MMA:                 []
Missing cids for UMCCONF_NSA_D3M3ML3:              []

Show code cell source Hide code cell source

# Create start years (data availability) plot

dfs = msm.check_startyears(dfd)
df_new = dfs.iloc[0]
df_new = df_new.to_frame()
df_new = df_new.reset_index()
df_new = df_new[~df_new["xcat"].isin(econ)]
df_new["cat"] = df_new["xcat"].apply(lambda x: x.split("_")[0])
df_new["adj"] = df_new["xcat"].apply(lambda x: x.split("_")[1])
df_new["trans"] = df_new["xcat"].apply(lambda x: x.split("_")[2])
df_new["col"] = df_new["cat"].apply(lambda x: cat_dict_to_column(x, cat_dict))
df_new2 = df_new
df_new2["col"] = df_new["cat"].apply(lambda x: cat_dict_to_key(x, cat_dict))

dfd.groupby(["xcat"])["cid"].nunique()[dfd.groupby(["xcat"])["cid"].nunique() > 1]

df_new = df_new.sort_values(by="USD")
df_new2 = df_new2.sort_values(by="USD")


def add_row(df):
    df_return = pd.DataFrame()
    for col_value in df["col"].unique():
        df_reduced = df[df["col"] == col_value]
        df_reduced.insert(0, "row", range(0, len(df_reduced)))
        df_return = pd.concat([df_return, df_reduced])
    return df_return


def reorder_cols(df):
    idea_count = []
    for idea in df.T.index:
        idea_count.append(df[idea].count())
    ideas = [*df.T.index]
    df_to_sort = pd.DataFrame(idea_count, ideas)
    df_sort = df_to_sort.sort_values(by=0)
    df_sort_index = [*df_sort.index]
    return df[df_sort_index]


df_new = add_row(df_new)
df_new2 = add_row(df_new2)

df_new2_pivot = df_new2.pivot(index="row", columns="col", values="USD")
df_new2_pivot = df_new2_pivot[order]
df_new2["label"] = (
    df_new["cat"]
    .str.cat(df_new["adj"].apply(lambda x: "\n" + x))
    .str.cat(df_new["trans"].apply(lambda x: "\n" + x))
    .str.cat(df_new2["USD"].apply("{:.0f}".format), sep="\n")
)
labels = df_new2.pivot(index="row", columns="col", values="label")
labels = labels[order]
header = "Start years of quantamental indicators."
sns.set(rc={"figure.figsize": (30, 25)})
sns.heatmap(
    df_new2_pivot, annot=labels, fmt="", cmap="YlOrBr", linewidths=10, yticklabels=[]
)
plt.tick_params(
    axis="both",
    which="major",
    labelsize=18,
    labelbottom=False,
    bottom=False,
    top=False,
    labeltop=True,
)
plt.xlabel("")
plt.ylabel("")
plt.title(header, fontsize=25, pad=50)
plt.show()

print("Last updated:", date.today())

Last updated: 2024-11-07

Show code cell source Hide code cell source

# Create missing days plot


def business_day_dif(df: pd.DataFrame, maxdate: pd.Timestamp):
    """
    Number of business days between two respective business dates.
    :param <pd.DataFrame> df: DataFrame cross-sections rows and category columns. Each
        cell in the DataFrame will correspond to the start date of the respective series.
    :param <pd.Timestamp> maxdate: maximum release date found in the received DataFrame.
        In principle, all series should have values up until the respective business
        date. The difference will represent possible missing values.
    :return <pd.DataFrame>: DataFrame consisting of business day differences for all
        series.
    """
    year_df = maxdate.year - df.apply(lambda x: x.dt.year)
    year_df *= 52

    week_max = maxdate.week
    week_df = week_max - df.apply(lambda x: x.dt.isocalendar().week)

    # Account for difference over a year.
    week_df += year_df

    # Account for weekends.
    week_df *= 2

    df = (maxdate - df).apply(lambda x: x.dt.days)
    return df - week_df


df_ends = dfd[["cid", "xcat", "real_date"]].groupby(["cid", "xcat"]).max()
df_ends["real_date"] = df_ends["real_date"].dt.strftime("%Y-%m-%d")
df_end = df_ends.unstack().loc[:, "real_date"]
df_end = df_end.apply(pd.to_datetime)
maxdate = df_end.max().max()
df_days = business_day_dif(df=df_end, maxdate=maxdate)

df_right = df_days.iloc[0].to_frame()
df_merged = df_new2.merge(df_right, on="xcat", how="inner")
df_merged = df_merged.reindex(df_new2.index)

df_days_trans = pd.DataFrame(
    [df_days.iloc[0].to_frame().index, df_days.iloc[0].to_frame().values]
).T
df_days_trans = df_days_trans.rename(columns={0: "xcat", 1: "days"})
df_days_trans["days"] = df_days_trans["days"].apply(int)

df_merged = df_new2.merge(df_days_trans, on="xcat", how="inner")
df_merged = df_merged.reindex(df_new2.index)
df_merged = df_merged.dropna()

df_new2_pivot = df_merged.pivot(index="row", columns="col", values="days")
df_new2_pivot = df_new2_pivot[order]
df_new2_pivot = df_new2_pivot.astype(float)
df_merged["label"] = (
    df_merged["cat"]
    .str.cat(df_merged["adj"].apply(lambda x: "\n" + x))
    .str.cat(df_merged["trans"].apply(lambda x: "\n" + x))
    .str.cat(df_merged["days"].apply("{0:.0f}".format), sep="\n")
)
labels = df_merged.pivot(index="row", columns="col", values="label")
labels = labels[order]
header = "Missing days of quantamental indicators."
sns.set(rc={"figure.figsize": (25, 25)})
sns.heatmap(
    df_new2_pivot, annot=labels, fmt="", cmap="YlOrBr", linewidths=10, yticklabels=[]
)
plt.tick_params(
    axis="both",
    which="major",
    labelsize=18,
    labelbottom=False,
    bottom=False,
    top=False,
    labeltop=True,
)
plt.xlabel("")
plt.ylabel("")
plt.title(header, fontsize=25, pad=50)
plt.show()

Average vintage grading is mixed, with most on the low-side. Some housing, labor and prices indicators are of the highest quality in terms of vintage grades.

Show code cell source Hide code cell source

# Create vintage grading chart

dfd_grad = dfd[dfd["real_date"].apply(lambda x: x.date()) > date(2000, 1, 3)]
dfd_grad = dfd_grad[["xcat", "cid", "real_date", "grading"]]
dfd_grad["grading"] = dfd_grad["grading"].astype(float).round(2)

df_ags = dfd_grad.drop(["cid"], axis=1).groupby(["xcat"]).mean()

df_merged = df_new2.merge(df_ags, on="xcat", how="left")
df_merged = df_merged.reindex(df_new2.index)
df_merged = df_merged.dropna()

df_new2_pivot = df_merged.pivot(index="row", columns="col", values="grading")
df_new2_pivot = df_new2_pivot[order]
df_new2_pivot = df_new2_pivot.astype(float)
df_merged["label"] = (
    df_merged["cat"]
    .str.cat(df_merged["adj"].apply(lambda x: "\n" + x))
    .str.cat(df_merged["trans"].apply(lambda x: "\n" + x))
    .str.cat(df_merged["grading"].apply("{0:.1f}".format), sep="\n")
)
labels = df_merged.pivot(index="row", columns="col", values="label")
labels = labels[order]
header = "Average grade of vintages since 2000-01-03."
sns.set(rc={"figure.figsize": (25, 25)})
sns.heatmap(
    df_new2_pivot, annot=labels, fmt="", cmap="YlOrBr", linewidths=10, yticklabels=[]
)
plt.tick_params(
    axis="both",
    which="major",
    labelsize=18,
    labelbottom=False,
    bottom=False,
    top=False,
    labeltop=True,
)
plt.xlabel("")
plt.ylabel("")
plt.title(header, fontsize=25, pad=50)
plt.show()

The main set of U.S. indicators informs on the economy with average time lags between 0 and 80 days from the end of the observed periods and between 20 and 300 days from the median date of the observed period.

Show code cell source Hide code cell source

# Create lag plots

df_range = dfd
df_range=dfd[~dfd["xcat"].isin(econ)].copy()
df_range["cat"] = df_range["xcat"].apply(lambda x: x.split("_")[0])
df_range["adj"] = df_range["xcat"].apply(lambda x: x.split("_")[1])
df_range["trans"] = df_range["xcat"].apply(lambda x: x.split("_")[2])
df_range["idea"] = df_range["cat"].apply(lambda x: cat_dict_to_key(x, cat_dict))
df_range["label"] = (
    df_range["cat"]
    .str.cat(df_range["adj"].apply(lambda x: "\n" + x))
    .str.cat(df_range["trans"].apply(lambda x: "\n" + x))
)


def group_mean_sd(df, val):
    mean = df.groupby(["xcat"])[val].mean()
    sd = df.groupby(["xcat"])[val].std()
    return mean, sd


df_mean_eop, df_sd_eop = group_mean_sd(df_range, "eop_lag")
order_eop = df_mean_eop.sort_values().index
df_mean_mop, df_sd_mop = group_mean_sd(df_range, "mop_lag")
order_mop = df_mean_mop.sort_values().index

sns.set_theme(style="darkgrid")
g = sns.barplot(
    y="xcat",
    x="eop_lag",
    data=df_range,
    ci="sd",
    order=order_eop,
    hue="idea",
    dodge=False,
)
plt.title("End of observation period lags", fontsize=25)
plt.show()

Show code cell source Hide code cell source

h = sns.barplot(
    y="xcat",
    x="mop_lag",
    data=df_range,
    ci="sd",
    order=order_mop,
    hue="idea",
    dodge=False,
)
plt.title("Median of observation period lags", fontsize=25)
plt.show()

History

Show code cell source Hide code cell source

# Create definition dictionary for ease in subsequent analysis

def_dict = {
    "ISMMANU_SA_3MMA": "ISM manufacturing survey, sa: 3mma",
    "ISMMANU_SA_D3M3ML3": "ISM manufacturing survey, sa: diff 3m/3m",
    "ISMSERV_SA_3MMA": "ISM services survey, sa: 3mma",
    "ISMSERV_SA_D3M3ML3": "ISM services survey, sa:diff 3m/3m",
    "PHILMANU_SA_3MMA": "Philadelphia manufacturing survey, sa: 3mma",
    "PHILMANU_SA_D3M3ML3": "Philadelphia manufacturing survey, sa: diff 3m/3m",
    "NYMANU_SA_3MMA": "Empire State manufacturing survey, sa: 3mma",
    "NYMANU_SA_D3M3ML3": "Empire State manufacturing survey, sa: diff 3m/3m",
    "DGORDERS_SA_P1M1ML1": "Durable goods orders, sa: %m/m",
    "DGORDERS_SA_P3M3ML3AR": "Durable goods orders, sa: %3m/3m",
    "DGORDERS_SA_P1M1ML12": "Durable goods orders, sa: %oya",
    "DGORDERS_SA_P1M1ML12_3MMA": "Durable goods orders, sa: %oya, 3mma",
    "DGORDERSXD_SA_P1M1ML1": "Durable goods orders ex defense, sa: %m/m",
    "DGORDERSXD_SA_P3M3ML3AR": "Durable goods orders ex defense, sa:  %3m/3m",
    "DGORDERSXD_SA_P1M1ML12": "Durable goods orders ex defense, sa:  %oya",
    "DGORDERSXD_SA_P1M1ML12_3MMA": "Durable goods orders ex defense, sa: %oya, 3mma",
    "BINVENTORIES_SA_P1M1ML1": "Business inventories, sa: %m/m",
    "BINVENTORIES_SA_P3M3ML3AR": "Business inventories, sa: %3m/3m",
    "BINVENTORIES_SA_P1M1ML12": "Business inventories, sa: %oya",
    "BINVENTORIES_SA_P1M1ML12_3MMA": "Business inventories, sa: %oya, 3mma",
    "CFNAI_SA_3MMA": "Chicago fed national activity index, sa: 3mma",
    "CFNAI_SA_D3M3ML3": "Chicago fed national activity index, sa: diff 3m/3m",
    "CBCCONF_SA_3MMA": "Conference board consumer confidence, sa: 3mma",
    "CBCCONF_SA_D3M3ML3": "Conference board consumer confidence, sa: diff 3m/3m",
    "UMCCONF_NSA_3MMA": "University of Michigan consumer sentiment, nsa: 3mma",
    "UMCCONF_NSA_D3M3ML3": "University of Michigan consumer sentiment, nsa: diff 3m/3m",
    "CONS_SA_P1M1ML12": "Real consumption, sa: %oya",
    "CONS_SA_P1M1ML12_3MMA": "Real consumption, sa: %oya, 3mma ",
    "CONS_SA_P3M3ML3AR": "Real consumption, sa: %3m/3m",
    "RSALES_SA_P1M1ML1": "Retail sales: total, sa, %m/m",
    "RSALES_SA_P6M6ML6AR": "Retail sales: total, sa, %6m/6m",
    "RSALES_SA_P1M1ML12": "Retail sales: total, sa, %oya",
    "RSALES_SA_P1M1ML12_3MMA": "Retail sales: total, sa ,%oya, 3mma",
    "RSALESXAF_SA_P1M1ML1": "Retail sales: ex autos and gas, sa",
    "RSALESXAF_SA_P6M6ML6AR": "Retail sales: ex autos and gas, sa, %6m/6m",
    "RSALESXAF_SA_P1M1ML12": "Retail sales: ex autos and gas, sa, %oya",
    "RSALESXAF_SA_P1M1ML12_3MMA": "Retail sales: ex autos and gas, sa, %oya, 3mma",
    "PERSINC_SA_P1M1ML12": "Personal income, sa: %oya",
    "PERSINC_SA_P1M1ML12_3MMA": "Personal income, sa: %oya, 3mma",
    "PERSINC_SA_P3M3ML3AR": "Personal income, sa: %3m/3m",
    "CARSALES_SA_P1M1ML12": "Moto vehicle sales, sa: %oya",
    "CARSALES_SA_P1M1ML12_3MMA": "Moto vehicle sales, sa: %oya, 3mma",
    "CONSTRSPEND_SA_P1M1ML12": "Construction spending, sa: %oya",
    "CONSTRSPEND_SA_P1M1ML12_3MMA": "Construction spending, sa: %oya, 3mma",
    "NAHBSURV_SA_3MMA": "Homebuilders survey, sa: 3mma",
    "NAHBSURV_SA_D3M3ML3": "Homebuilders survey, sa: diff 3m/3m",
    "NEWHOMESALES_SA_P1M1ML12": "Home sales: new, sa, %oya",
    "NEWHOMESALES_SA_P1M1ML12_3MMA": "Home sales: new, sa, %oya, 3mma",
    "EXISTHOMESALES_SA_P1M1ML12": "Home sales: existing, sa, %oya",
    "EXISTHOMESALES_SA_P1M1ML12_3MMA": "Home sales: existing, sa, %oya, 3mma",
    "PENDHOMESALES_SA_P1M1ML12": "Pending homes sales: sa, %oya",
    "PENDHOMESALES_SA_P1M1ML12_3MMA": "Pending homes sales: sa,%oya, 3mma",
    "INJLCLAIMS_SA_D4W4WL4": "Initial jobless claims, sa: dif 4w/4w",
    "INJLCLAIMS_SA_D4W4WL52": "Initial jobless claims, sa: dif oya, 4wma",
    "INJLCLAIMS_SA_4WMAv5YMM": "Initial jobless claims, sa: %4wma vs 5yma",
    "INJLCLAIMS_SA_4WMAv10YMM": "Initial jobless claims, sa: %4wma vs 10yma",
    "COJLCLAIMS_SA_D4W4WL4": "Continuing jobless claims, sa: dif 4w/4w",
    "COJLCLAIMS_SA_D4W4WL52": "Continuing jobless claims, sa: dif oya, 4wma",
    "COJLCLAIMS_SA_4WMAv5YMM": "Continuing jobless claims, sa: %4wma vs 5yma",
    "COJLCLAIMS_SA_4WMAv10YMM": "Continuing jobless claims, sa: %4wma vs 10yma",
    "NFPAYROLLPRIV_SA_P1M1ML1": "Non-farm payroll, private sector, sa: %m/m",
    "NFPAYROLLPRIV_SA_P3M3ML3": "Non-farm payroll, private sector, sa: %3m/3m",
    "NFPAYROLLPRIV_SA_P1M1ML12": "Non-farm payroll, private sector, sa: %oya",
    "NFPAYROLLPRIV_SA_P1M1ML12_3MMA": "Non-farm payroll, private sector, sa: %oya, 3mma",
    "ADPEMPL_SA_P1M1ML1": "ADP employment, sa: %m/m",
    "ADPEMPL_SA_P3M3ML3AR": "ADP employment, sa: %3m/3m",
    "ADPEMPL_SA_P1M1ML12": "ADP employment, sa: %oya",
    "ADPEMPL_SA_P1M1ML12_3MMA": "ADP employment, sa: %oya, 3mma",
    "HOURLYWAGES_SA_P1M1ML1": "Hourly wages, sa: %m/m",
    "HOURLYWAGES_SA_P3M3ML3AR": "Hourly wages, sa: %3m/3m",
    "HOURLYWAGES_SA_P1M1ML12": "Hourly wages, sa: %oya",
    "HOURLYWAGES_SA_P1M1ML12_3MMA": "Hourly wages, sa: %oya, 3mma",
    "HOURLYCOMP_SA_P1Q1QL4": "Hourly compensation, sa: % oya",
    "HOURLYPROD_SA_P1Q1QL4": "Labor productivity, sa: % oya",
    "CPIUH_SA_P1M1ML1": "Consumer price index, sa: %m/m",
    "CPIUH_SA_P6M6ML6AR": "Consumer price index, sa: %6m/6m",
    "CPIUH_NSA_P1M1ML12": "Consumer price index, nsa: %oya",
    "CPIUH_NSA_P1M1ML12_3MMA": "Consumer price index, nsa: %oya, 3mma",
    "CPIUC_SA_P1M1ML1": "Core CPI, sa: %m/m",
    "CPIUC_SA_P6M6ML6AR": "Core CPI, sa: %6m/6m",
    "CPIUC_NSA_P1M1ML12": "Core CPI, nsa: %oya",
    "CPIUC_NSA_P1M1ML12_3MMA": "Core CPI, nsa: %oya, 3mma",
    "PPIH_SA_P1M1ML1": "Producer price index, sa: %m/m",
    "PPIH_SA_P6M6ML6AR": "Producer price index, sa: %6m/6m",
    "PPIH_NSA_P1M1ML12": "Producer price index, nsa: %oya",
    "PPIH_NSA_P1M1ML12_3MMA": "Producer price index, nsa: %oya, 3mma",
    "PPIC_SA_P1M1ML1": "Core PPI, sa: %m/m",
    "PPIC_SA_P6M6ML6AR": "Core PPI, sa: %6m/6m",
    "PPIC_NSA_P1M1ML12": "Core PPI, nsa: %oya",
    "PPIC_NSA_P1M1ML12_3MMA": "Core PPI, nsa: %oya, 3mma",
    "IMPIH_NSA_P1M1ML12": "Import price index, nsa: %oya",
    "IMPIH_NSA_P1M1ML12_3MMA": "Import price index, nsa: %oya, 3mma",
    "IMPIC_NSA_P1M1ML12": "Core import price index, nsa: %oya",
    "IMPIC_NSA_P1M1ML12_3MMA": "Core import price index, nsa: %oya, 3mma",
}

Business surveys

Business confidence indices track broadly similar cyclical dynamics, but with different absolute and relative amplitudes. Industry and services sentiment can at times deviate notably.

xcatx = ["ISMSERV", "ISMMANU", "PHILMANU", "NYMANU"]
xcatxx = [xc + "_SA_3MMA" for xc in xcatx]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="U.S. main business survey levels, daily information states",
    xcat_labels=labels,
    start="1995-01-01",
    title_adj=1.05,
    ncol=1,
    legend_fontsize=10,
    size=(10, 5),
)

xcatx = ["ISMSERV", "ISMMANU", "PHILMANU", "NYMANU"]
xcatxx = [xc + "_SA_D3M3ML3" for xc in xcatx]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="U.S. main business surveys, sa, diff 3m/3m, daily information states",
    xcat_labels=labels,
    title_adj=1.05,
    legend_fontsize=10,
    start="1995-01-01",
    size=(10, 5),
)

Composite activity indicator

Variations in the information states of the Chicago Fed activity index have been dominated by crises and recoveries. Mini-cycles in activity seem to play only a minor role.

xcatxx = ["CFNAI_SA_3MMA"]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Chicago Fed National Activity Index, sa, information states",
    xcat_labels=labels,
    title_adj=1.05,
    legend_fontsize=10,
    start="2001-01-01",
    size=(10, 5),
)

Orders and inventories

Order growth, excluding defense, has naturally been less prone to outliers that headline durable good orders. Inventories growth seems to undergo much milder cycles and respond to order trends with some lag.

xcatx = ["DGORDERS", "DGORDERSXD", "BINVENTORIES"]
xcatxx = [xc + "_SA_P1M1ML12_3MMA" for xc in xcatx]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="U.S. durable goods orders and inventories, %oya, 3mma, daily information states",
    xcat_labels=labels,
    title_adj=1.05,
    legend_fontsize=10,
    start="1990-01-01",
    size=(10, 5),
)

Private consumption

Consumer confidence

The two most prominent consumer surveys in the U.S. have naturally tracked the same cycles, but dynamics have at times sent very different signals to the markets, in terms of short-term dynamics and levels relative to a long-term average.

xcatxx = ["CBCCONF_SA_3MMA", "UMCCONF_NSA_3MMA"]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="U.S. consumer confidence indicators, daily information states",
    xcat_labels=labels,
    title_adj=1.05,
    legend_fontsize=10,
    start="1980-01-01",
    size=(10, 5),
)

Income and spending

Depending on economic circumstances, income and personal consumption dynamics can deviate vastly. For example, during the initial year of the COVID crisis income growth accelerated, while consumption collapsed.

xcatxx = ["CONS_SA_P1M1ML12", "PERSINC_SA_P1M1ML12"]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Personal consumption and income growth, daily information states",
    xcat_labels=labels,
    title_adj=1.05,
    legend_fontsize=10,
    start="2000-01-01",
    size=(10, 5),
)

Retail sales value growth has been a lot more stable, excluding autos and gas.

xcatxx = ["RSALES_SA_P1M1ML12", "RSALESXAF_SA_P1M1ML12"]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Retail sales (value), %oya, daily information states",
    xcat_labels=labels,
    title_adj=1.05,
    legend_fontsize=10,
    start="1994-01-01",
    size=(10, 5),
)

Car sales (units) have tracked the broad cycles of retail sales, but at a different pace, excluding inflation effects, and with some sector-specific dynamics.

xcatxx = ["CARSALES_SA_P1M1ML12", "CARSALES_SA_P1M1ML12_3MMA"]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Motor vehicle sales, %oya, daily information states",
    title_adj=1.05,
    legend_fontsize=10,
    xcat_labels=labels,
    start="1994-01-01",
    size=(10, 5),
)

Housing indicators

Construction spending has displayed a pronounced cyclical pattern over the past three decades with a particularly powerful boom-bust cycle 2004-2012.

xcatxx = ["CONSTRSPEND_SA_P1M1ML12", "CONSTRSPEND_SA_P1M1ML12_3MMA"]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Construction spending, %oya, daily information states",
    title_adj=1.05,
    legend_fontsize=10,
    xcat_labels=labels,
    start="1994-01-01",
    size=(10, 5),
)

The homebuilders survey has been tracking the broad cycles in cunstruction, but not very closely and has not always been a leading indicator.

xcatxx = ["NAHBSURV_SA_3MMA"]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Homebuilders survey, sa, 3mma, daily information states",
    title_adj=1.05,
    legend_fontsize=10,
    start="1994-01-01",
    size=(10, 5),
)

Among the home sales data, new home sales seem to post the most erratic growth rates. Exisiting and pending home sales growth have recored similar dynamics, with pending home sales information sometimes leading.

xcatx = ["NEWHOMESALES", "EXISTHOMESALES", "PENDHOMESALES"]
xcatxx = [xc + "_SA_P1M1ML12_3MMA" for xc in xcatx]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Home sales, %oya, 3mma, daily information states",
    legend_fontsize=10,
    title_adj=1.05,
    xcat_labels=labels,
    start="2000-01-01",
    size=(10, 5),
)

Labor market

Jobless claims

Jobless claims rise faster in downturns than they decline in recoveries and expansions.

xcatx = ["INJLCLAIMS_SA", "COJLCLAIMS_SA"]
xcatxx = [xc + "_4WMAv10YMM" for xc in xcatx]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Jobless claims, 4wma, sa, % versus 10-year moving median, daily information states",
    xcat_labels=labels,
    legend_fontsize=10,
    title_adj=1.05,
    start="1990-01-01",
    size=(10, 5),
)

Continuous claims have been a more stable and only slightly lagging indicator of short-term trends in claims.

dfxx = dfd.copy()
filt = dfxx["value"] < -600000
dfxx.loc[filt, "value"] = -600000
filt = dfxx["value"] > 600000
dfxx.loc[filt, "value"] = 600000
xcatx = ["INJLCLAIMS_SA", "COJLCLAIMS_SA"]
xcatxx = [xc + "_D4W4WL4" for xc in xcatx]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfxx,
    xcats=xcatxx,
    cids=["USD"],
    title="Jobless claims, sa, dif 4w/4w (winsorized at 600k), daily information states",
    xcat_labels=labels,
    legend_fontsize=10,
    title_adj=1.05,
    start="1990-01-01",
    size=(10, 5),
)

Employment growth

Annual private employment growth has been a stable business cycle indicator, albeit subject to great volatility during the COVID pandemic.

xcatx = ["NFPAYROLLPRIV", "ADPEMPL"]
xcatxx = [xc + "_SA_P1M1ML12" for xc in xcatx]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Employment growth, %oya, daily information states",
    xcat_labels=labels,
    legend_fontsize=10,
    title_adj=1.05,
    start="1990-01-01",
    size=(10, 5),
)

Labor cost and compensation

Wage growth (3-month averages) has since 2007 been a far more stable indicator on labor cost trends then compensation (quarterly).

xcatxx = ["HOURLYWAGES_SA_P1M1ML12_3MMA", "HOURLYCOMP_SA_P1Q1QL4"]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Hourly wages and compensation, %oya, 3mma or quarterly, daily information states",
    xcat_labels=labels,
    title_adj=1.05,
    legend_fontsize=10,
    start="2000-01-01",
    size=(10, 5),
)

Prices

On the consumer price side, core inflation information has been a lot more stable than headline CPI inflation data.

xcatxx = ["CPIUH_NSA_P1M1ML12", "CPIUC_NSA_P1M1ML12"]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="CPI, headline and core, %oya, daily information states",
    xcat_labels=labels,
    title_adj=1.05,
    legend_fontsize=10,
    start="2000-01-01",
    size=(10, 5),
)

Import price inflation has been a lot more volatile than consumer and producer prices.

xcatxx = ["CPIUH_NSA_P1M1ML12", "PPIH_NSA_P1M1ML12", "IMPIH_NSA_P1M1ML12"]
labels = [def_dict[key] for key in xcatxx]
msp.view_timelines(
    dfd,
    xcats=xcatxx,
    cids=["USD"],
    title="Headline price indices, %oya, daily information states",
    xcat_labels=labels,
    title_adj=1.05,
    legend_fontsize=10,
    start="2000-01-01",
    size=(10, 5),
)

Importance

Relevant research

“Empirical evidence suggests that investors pay less attention to macroeconomic news when market sentiment is positive. Market responses to economic data surprises have historically been muted in high sentiment periods. Behavioral research supports the idea that investors prefer heuristic decision-making and neglect fundamental information in bullish markets, but pay more attention in turbulent times. This allows prices to diverge temporarily from fundamentals and undermines the conventional risk-return trade-off when sentiment is high. Low-risk portfolios tend to outperform subsequently. The sentiment bias also means that fundamental predictors of market prices work better in low-sentiment periods than in high-sentiment periods.” Macrosynergy

Empirical clues

There is some evidence of a negative predictive relation between the Chicago activity index and subsequent Treasury futures returns, in line with the standard hypothesis that high activity growth triggers monetary policy tightening and low activity growth monetary easing. Predictive power is even stronger if one caps (winsorizes) outliers of index values.

cidx = ["USD"]
sdate = "2001-01-01"

cr = msp.CategoryRelations(
    dfd,
    xcats=["CFNAI_SA_3MMA", "GBF10YXR_NSA"],
    cids=cidx,
    freq="M",
    lag=1,
    xcat_aggs=["last", "sum"],
    fwin=1,
    start=sdate,
    years=None,
)

cr.reg_scatter(
    labels=False,
    coef_box="upper right",
    prob_est="map",
    title = "Chicago Fed activity index (information states) and U.S. Treasury future returns, since 2001", 
    xlab="Chicago Fed national activity index, seasonally adjusted, 3 month moving average",
    ylab="10-year Treasury future returns, next month",
)

cidx = ["USD"]
sdate = "2001-01-01"

cr = msp.CategoryRelations(
    dfd,
    xcats=["CFNAI_SA_3MMA", "GBF10YXR_NSA"],
    cids=cidx,
    freq="M",
    lag=1,
    xcat_aggs=["mean", "sum"],
    fwin=1,
    start=sdate,
    years=None,
    xcat_trims=[3, 100],
)

cr.reg_scatter(
    labels=False,
    coef_box="upper right",
    prob_est="map",
    title = "Trimmed Chicago Fed activity index (information state) and U.S. Treasury future returns, since 2001", 
    xlab="Chicago Fed national activity index, seasonally adjusted, 3 month moving average, winsorized at 3",
    ylab="10-year Treasury future returns, next month",
)