Dashboard #

Get packages and JPMaQS data #

This notebook primarily relies on the standard packages available in the Python data science stack. However, there is an additional package, macrosynergy that is required for two purposes:

  • Downloading JPMaQS data: The macrosynergy package facilitates the retrieval of JPMaQS data, which is used in the notebook.

  • For the analysis of quantamental data and value propositions: The macrosynergy package provides functionality for performing quick analyses of quantamental data and exploring value propositions.

For detailed information and a comprehensive understanding of the macrosynergy package and its functionalities, please refer to the “Introduction to Macrosynergy package” notebook on the Macrosynergy Quantamental Academy or visit the following link on Kaggle .

# Run only if needed!
"""
%%capture
! pip install macrosynergy --upgrade"""
import numpy as np
import pandas as pd
from pandas import Timestamp
import matplotlib.pyplot as plt
import matplotlib.colors as mcolors
from matplotlib.colors import LinearSegmentedColormap
import seaborn as sns
import os

import macrosynergy.management as msm
import macrosynergy.panel as msp
import macrosynergy.signal as mss
import macrosynergy.pnl as msn

from macrosynergy.download import JPMaQSDownload
pd.set_option('display.width', 400)
import warnings

warnings.simplefilter("ignore")
C:\Users\GlennRegis\AppData\Roaming\Python\Python38\site-packages\pandas\core\computation\expressions.py:21: UserWarning: Pandas requires version '2.7.3' or newer of 'numexpr' (version '2.7.1' currently installed).
  from pandas.core.computation.check import NUMEXPR_INSTALLED

Framework #

We create a dashboard which shows z-scored Macrosynergy indicators as of the latest trading date in a heatmap. We choose the indicators that we believe have the largest effect on government bond returns. The indicators we choose are inflation, primary balance, borrowing requirements and intuitive GDP.

Literature #

For inflation, we use MS’s expected inflation indicator. We draw upon the literature on fixed income so as to understand the expected direction of our indicator. The Bank of International Settlements shows proof of positive inflation risk premias, such that the higher the the risk premia, the higher the yields. Thus, we expect there to be a negative relation between our expected inflation indicator and government bond returns.

As per our Macrosynergy research , we find a negative relation between borrowing requirements and government bond returns.

With regards to the primary balance, the IMF proves that higher deficits and public debt lead to a significant increase in long-term interest rates. Thus, here too we expect to find e negative relationship between our primary balance.

With regards to intuitive GDP, the relationship is more complex. Our own Macrosynergy research shows a positive predictive effect of GDP on real interest rates. In addition, we have also found a positive relation between real yields and government bond returns. By combining the two, we can expect a positive relation between intuitive GDP and government bond returns.

Methodology #

Once the z-scores are built, we show an initial heatmap of those, windsorised at 3 standard deviations.

Subsequently, we study the actual correlations of our end of month indicator values with end of month government bond returns throughout the whole history. Once we have these correlations, we apply a sign adjustment to the z-scores. If we expect a negative relationship, as per the literature above, with the returns, we will apply a negative sign and viceversa.

The final heatmap will show the sign-adjusted heatmap.

# Bond-specific cross-sections

cids_dmea = ["FRF", "DEM", "ITL", "ESP", "EUR"]

cids_dmxe = [ "CHF", "GBP", "JPY", "SEK","USD"]

cids_dm = cids_dmea + cids_dmxe
cids_g10 = ["AUD", "DEM", "FRF", "ESP", "ITL", "JPY", "NZD", "GBP", "USD"]

cids_latm = ["BRL", "CLP", "COP", "MXN", "PEN" ]  # Latam sovereigns
cids_emea = ["CZK",
"HUF",
"ILS",
"PLN",
    "RON",
    "ZAR",
    "TRY",
]  # EMEA sovereigns
cids_emas = [
   
    "CNY",
    "IDR",
    "KRW",
    "MYR",
    "PHP",
    "THB",
]  # EM Asia sovereigns

cids_ea = ["DEM", "FRF", "ESP", "ITL"]  # major Euro currencies before EUR


cids_em = cids_emea + cids_latm + cids_emas
cids = cids_dm + cids_em
cat_mapping = {
    'CPIH_SA_P1M1ML12_ZN': 'CPI zn',
    'CPIH_SA_P1M1ML12': 'CPI',
    'INFE1Y_JA_ZN': 'Inflation expectations zn',
    'INFE1Y_JA': 'Inflation expectations',
    'RGDP_SA_P1Q1QL1AR_ZN': 'Real GDP zn',
    'INTRGDP_NSA_P1M1ML12_3MMA_ZN': 'Intuitive GDP zn',
    'INTRGDP_NSA_P1M1ML12_3MMA': 'Intuitive GDP',
    'WAGES_NSA_P1M1ML12_ZN': 'Wages zn',
    'CABGDPRATIO_NSA_12MMA_ZN': 'Current Account Balance zn',
    'GGOBGDPRATIO_NSA_ZN' : 'Overall Balance zn',
    'GGOBGDPRATIO_NSA' : 'Overall Balance',
    'GGPBGDPRATIO_NSA_ZN': 'Primary Balance zn',
    'GGPBGDPRATIO_NSA': 'Primary Balance', 
    'GNBRGDP_NSA_ZN': 'Net Borrowing Requirements zn',
    'GNBRGDP_NSA': 'Net Borrowing Requirements',    
    'XGGPBGDPRATIO_NSA_ZN': 'Excess Primary Balance zn',
    'NIIPGDP_NSA_ZN': 'Net international investment position zn',
    'CRESFXGDP_NSA_D1M1ML1_ZN': 'Currency Reserves zn',
    'RIR_NSA_ZN': 'Real interest rate zn',
    'DU02YETP_NSA_ZN': 'Duration term premia 2-year zn',
    'CDS02YXRxEASD_NSA_ZN': '2 year CDS return volatility zn',
    'DU02YXRxEASD_NSA_ZN': '2 year swap volatility zn',
    'EQXRxEASD_NSA_ZN': 'Equity return volatility zn',
    'GB10YXRxEASD_NSA_ZN': 'Government Bond Return Volatility zn',
    'CDS05YSPRD_NSA_ZN': '5-year CDS spreads zn',
    'DU05YXR_NSA': '5-year swap returns',
    'GB05YR_NSA': '5-year govy returns',
    'CDS05YXR_NSA': '5-year CDS returns'
}


macro_mapping = [
    'Inflation expectations zn',
#     'Effective inflation',
#    'Real GDP zn',
    'Intuitive GDP zn',
#     'Wages',
]

fiscal_mapping = [
#     'Current Account Balance', 
     'Primary Balance zn',
    'Overall Balance zn',
#     'Net Borrowing Requirements',
    'Net Borrowing Requirements zn',
#     'Excess Primary Balance',
#     'Net international investment position',
]

The description of each JPMaQS category is available under Macro quantamental academy , or JPMorgan Markets (password protected). For tickers used in this notebook see Government debt sustainability , Sovereign CDS returns , Equity index future returns , and Government bond returns

# Category tickers

π = ['INFE1Y_JA',] # 'CPIH_SA_P1M1ML12', 
gdp = ['INTRGDP_NSA_P1M1ML12_3MMA']
wages = ['WAGES_NSA_P1M1ML12']
ex_bal = ['CABGDPRATIO_NSA_12MMA']
gov_ob = ['GGOBGDPRATIO_NSA']
bor_reqs = ['GNBRGDP_NSA']
xpb = ['XGGPBGDPRATIO_NSA']
nip = ['NIIPGDP_NSA']
mp = ['CRESFXGDP_NSA_D1M1ML1']
rir = ['RIR_NSA']
dur_vol = ['DU02YXRxEASD_NSA']
eq_vol = ['EQXRxEASD_NSA']
gb_vol = ['GB10YXRxEASD_NSA']
cds_vol = ['CDS02YXRxEASD_NSA']
cds = ['CDS05YSPRD_NSA']
tp = ['DU02YETP_NSA']

macro = π + gdp# + wages
fiscal = gov_ob + bor_reqs # + xpb + nip ex_bal 
mon_pol = mp + rir + tp
vol = cds_vol + dur_vol + eq_vol + gb_vol + cds

main = macro + fiscal #+ mon_pol + vol

# Target returns

rets = ['DU05YXR_NSA', 'GB05YR_NSA', 'CDS05YXR_NSA']

# Tickers

xcats = main + rets

tickers = [cid + "_" + xcat for cid in cids for xcat in xcats]
print(f"Maximum number of tickers is {len(tickers)}")
Maximum number of tickers is 196
# Download series from J.P. Morgan DataQuery by tickers

client_id: str = os.getenv("DQ_CLIENT_ID")
client_secret: str = os.getenv("DQ_CLIENT_SECRET")

with JPMaQSDownload(client_id=client_id, client_secret=client_secret) as dq:
    df = dq.download(
        tickers=tickers,
        start_date="2000-01-01",
        suppress_warning=True,
        metrics=[
            "value",
        ],
        show_progress=True,
    )
Downloading data from JPMaQS.
Timestamp UTC:  2024-04-16 13:05:25
Connection successful!
Requesting data: 100%|█████████████████████████████████████████████████████████████████| 10/10 [00:02<00:00,  4.61it/s]
Downloading data: 100%|████████████████████████████████████████████████████████████████| 10/10 [00:09<00:00,  1.05it/s]
Some expressions are missing from the downloaded data. Check logger output for complete list.
45 out of 196 expressions are missing. To download the catalogue of all available expressions and filter the unavailable expressions, set `get_catalogue=True` in the call to `JPMaQSDownload.download()`.
Some dates are missing from the downloaded data. 
6339 out of 6339 dates are missing.

The JPMaQS indicators we consider are downloaded using the J.P. Morgan Dataquery API interface within the macrosynergy package. This is done by specifying ticker strings, formed by appending an indicator category code to a currency area code <cross_section>. These constitute the main part of a full quantamental indicator ticker, taking the form DB(JPMAQS,<cross_section>_<category>,<info>) , where denotes the time series of information for the given cross-section and category. The following types of information are available:

value giving the latest available values for the indicator eop_lag referring to days elapsed since the end of the observation period mop_lag referring to the number of days elapsed since the mean observation period grade denoting a grade of the observation, giving a metric of real-time information quality.

After instantiating the JPMaQSDownload class within the macrosynergy.download module, one can use the download(tickers,start_date,metrics) method to easily download the necessary data, where tickers is an array of ticker strings, start_date is the first collection date to be considered and metrics is an array comprising the times series information to be downloaded. For more information see here

dfx = df.copy().sort_values(["cid", "xcat", "real_date"])
dfx.info()
print(dfx['xcat'].unique())
dfx_rets = dfx[dfx['xcat'].isin(rets)]
<class 'pandas.core.frame.DataFrame'>
Int64Index: 869362 entries, 0 to 869354
Data columns (total 4 columns):
 #   Column     Non-Null Count   Dtype         
---  ------     --------------   -----         
 0   real_date  869362 non-null  datetime64[ns]
 1   cid        869362 non-null  object        
 2   xcat       869362 non-null  object        
 3   value      869362 non-null  float64       
dtypes: datetime64[ns](1), float64(1), object(2)
memory usage: 33.2+ MB
['CDS05YXR_NSA' 'GGOBGDPRATIO_NSA' 'INFE1Y_JA' 'INTRGDP_NSA_P1M1ML12_3MMA'
 'DU05YXR_NSA' 'GNBRGDP_NSA' 'GB05YR_NSA']

The following cell fills in the Inflation expectation indicator for the primary Eurozone currencies [‘DEM’, ‘ESP’, ‘FRF’, ‘ITL’]. Essentially, it duplicates the EUR values and assigns them to their respective currencies.

cids_ea = ["DEM", "FRF", "ESP", "ITL"]
cidx = cids_ea
xcatx = ["INFE1Y_JA"]

calcs = [f"{xc}=  iEUR_{xc} + GGOBGDPRATIO_NSA - GGOBGDPRATIO_NSA" for xc in xcatx]


dfa = msp.panel_calculator(dfx, calcs=calcs, cids=cidx)
dfx = msm.update_df(dfx, dfa)

Transformations and checks #

Z-score calculation #

xcatx = main
cidx = cids

dfcorr = pd.DataFrame(columns=list(dfx.columns))

for xc in xcatx:
    dfaa = msp.make_zn_scores(
        dfx,
        xcat=xc,
        cids=cidx,
        sequential=True,
        min_obs=261 * 5,
        neutral="mean",
        pan_weight=0,
        thresh=3,
        postfix="_ZN",
        est_freq="m",
    )
    dfcorr = msm.update_df(dfcorr, dfaa)

dfx = msm.update_df(dfx, dfcorr)
### Mapping categories
main_zn = [x+'_ZN'for x in main] + rets
last_date = pd.Timestamp.today().date() - pd.offsets.BDay(n=1)
macro_zn = [x+'_ZN'for x in macro]
fiscal_zn = [x+'_ZN'for x in fiscal]
# We map the categories to more readable names
dfx['xcat'] = dfx['xcat'].map(cat_mapping)

macro_zn = [cat_mapping.get(cat, cat) for cat in macro_zn]
cats = macro_mapping + fiscal_mapping
# We split data between developed and emerging markets
dfx_dm = dfx[dfx['cid'].isin(list(set(cids_g10).intersection(cids_dm)))]
dfx_em = dfx[dfx['cid'].isin(list(set(cids_g10).intersection(cids_em)))]

# We create our signals for DM and EM
signals_dm = dfx_dm.loc[dfx_dm.xcat.isin(cats) & (dfx_dm.real_date == last_date)].pivot(index="xcat", columns="cid", values="value").sort_index()
signals_em = dfx_em.loc[dfx_em.xcat.isin(cats) & (dfx_em.real_date == last_date)].pivot(index="xcat", columns="cid", values="value").sort_index()

#print(signals.loc[cids_dm, :].T)
# Prelimiary heatmap NOT adjusted by expected direction. DM
# Create a custom colormap from green to red
colors = ["#d62728",  "white", "#4CBB17"]  # Green to red
n_bins = 20  # Increase this number to have more fine transition in the color
cmap_name = "custom1"
custom_cmap = LinearSegmentedColormap.from_list(cmap_name, colors, N=n_bins)


# coolwarm = plt.cm.get_cmap('coolwarm')
# inverted_coolwarm = mcolors.ListedColormap(coolwarm(np.linspace(1, 0, 256)))
# Plotting the heatmap
plt.figure(figsize=(14, 12))  # Adjust the figure size as necessary
sns.heatmap(signals_dm, annot=True, cmap=custom_cmap, center=0)
plt.title('Correlation heatmap of economic indicators - Developed markets')
plt.xticks(rotation=45, ha='right')
plt.show()
https://macrosynergy.com/notebooks.build/trading-factors/dashboard/_images/a7d3e16678a1c421530c65fc685be651c1e126f5af7923cd68a62dae101d4f56.png
# # Prelimiary heatmap NOT adjusted by expected direction. EM

# coolwarm = plt.cm.get_cmap('coolwarm')
# inverted_coolwarm = mcolors.ListedColormap(coolwarm(np.linspace(1, 0, 256)))
# # Plotting the heatmap
# plt.figure(figsize=(14, 12))  # Adjust the figure size as necessary
# sns.heatmap(signals_em, annot=True, cmap=inverted_coolwarm, center=0)
# plt.title('Correlation heatmap of economic indicators - Developed markets')
# plt.xticks(rotation=45, ha='right')
# plt.show()

Calculations for subsequent end of month bond returns #

dfxx = dfx.copy()

dfxx['real_date'] = pd.to_datetime(dfxx['real_date'])
# We separate returns from the macro data
dfx_rets = dfxx[dfxx['xcat'] == '5-year govy returns']
dfx_x = dfxx.loc[dfxx.xcat.isin(cats)].reset_index(drop=True)

# Filter DataFrames and calculate sum of returns and end of month values for the macro data
df_eom = dfx_x.groupby(['cid', 'xcat', pd.Grouper(key='real_date', freq='M')]).agg('last').reset_index()
df_sum = dfx_rets.groupby(['cid', 'xcat', pd.Grouper(key='real_date', freq='M')]).agg({'value': 'sum'}).reset_index()
rets = df_sum.copy()
rets['nxt_ret'] = df_sum.groupby(['cid', 'xcat'])['value'].shift(-1).reset_index(drop=True)

# check we're dealing with datetime objects
df_eom['real_date'] = pd.to_datetime(df_eom['real_date'])
rets['real_date'] = pd.to_datetime(rets['real_date'])
# Create new dataframe with all cats needed to calculate correlations
unique_cats = df_eom['xcat'].unique()
rets_exp_list = []
for cat in unique_cats:
    rets_copy = rets.copy()
    rets_copy['xcat'] = cat
    rets_exp_list.append(rets_copy)

rets_exp = pd.concat(rets_exp_list, ignore_index=True)
merged_data = pd.merge(df_eom, rets_exp[['real_date', 'cid', 'xcat', 'nxt_ret']],
                       on=['real_date', 'cid', 'xcat'], how='left').reset_index(drop=True)

merged_data = merged_data.dropna(subset=['value', 'nxt_ret'])

Correlation studies #

# We study the correlations between our indicators and the subsequent month returns
correlation_results = merged_data.groupby(['cid', 'xcat']).apply(
    lambda group: group[['value', 'nxt_ret']].corr().iloc[0, 1]
).reset_index().rename(columns={0: 'correlation'})

# Display the correlation results
correlation_results.sort_values(by=['xcat', 'cid'])
cid xcat correlation
0 DEM Inflation expectations zn -0.013778
4 ESP Inflation expectations zn -0.141032
8 FRF Inflation expectations zn -0.031134
12 GBP Inflation expectations zn 0.046769
16 ITL Inflation expectations zn -0.087661
20 JPY Inflation expectations zn 0.017440
23 USD Inflation expectations zn -0.004716
1 DEM Intuitive GDP zn -0.033754
5 ESP Intuitive GDP zn -0.183018
9 FRF Intuitive GDP zn -0.182295
13 GBP Intuitive GDP zn -0.092080
17 ITL Intuitive GDP zn -0.207908
21 JPY Intuitive GDP zn -0.017974
24 USD Intuitive GDP zn -0.007867
2 DEM Net Borrowing Requirements zn -0.016393
6 ESP Net Borrowing Requirements zn -0.174314
10 FRF Net Borrowing Requirements zn -0.084528
14 GBP Net Borrowing Requirements zn 0.138715
18 ITL Net Borrowing Requirements zn 0.029091
25 USD Net Borrowing Requirements zn -0.043416
3 DEM Overall Balance zn 0.060806
7 ESP Overall Balance zn -0.103449
11 FRF Overall Balance zn -0.021874
15 GBP Overall Balance zn -0.041571
19 ITL Overall Balance zn 0.058855
22 JPY Overall Balance zn -0.031613
26 USD Overall Balance zn 0.071351
# We change the signs so that positive sign in the z-score means positive for returns according to the correlations above
correlation_matrix = correlation_results.pivot(index='xcat', columns='cid', values='correlation')
correlation_matrix = correlation_matrix.reindex_like(signals_dm)
signals_dm_updated = signals_dm.multiply(correlation_matrix.apply(np.sign))
coolwarm = plt.cm.get_cmap('coolwarm')
inverted_coolwarm = mcolors.ListedColormap(coolwarm(np.linspace(1, 0, 256)))
# Plotting the heatmap
plt.figure(figsize=(14, 12))  # Adjust the figure size as necessary
sns.heatmap(signals_dm_updated, annot=True, cmap=custom_cmap, center=0)
plt.title('Sign-adjusted heatmap of economic indicators - Developed markets')
plt.xticks(rotation=45, ha='right')
plt.show()
https://macrosynergy.com/notebooks.build/trading-factors/dashboard/_images/6fd1c60b5a3ba8ade77cdae2bf600610fde3463f946d6fac5d929692bf5eaa51.png