You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With the current code in utils.py weighted deciles cannot be created if too many filers have 0 or negative values for the relevant income measure (e.g. expanded income). The add_quantile_table_row_variable() function creates the decile bins in two steps:
After sorting the records by the relevant income measure, the records are broken up into 10 bins with equal number of people or filing units.
If decile_details=True, which is the default option for difference and distribution tables, the original deciles are further broken down to include a group of filers with negative income and a group with 0 income.
The problem arises if more than 10% of records have <=0 income since the bin sizes will no longer increase monotonically and pandas will throw an error when calling pd.cut().
This exact problem arose in #2444 when trying to run recipe05 with the new CPS. Since recipe05 replaces expanded income with "market income" (i.e. does not include benefits received), the number of filers with <=0 income grew bigger than 10% of the records.
A short term solution would include modifying recipe05. A more permanent solution would involve modifying the bin creation in utils.py.
The text was updated successfully, but these errors were encountered:
With the current code in
utils.py
weighted deciles cannot be created if too many filers have 0 or negative values for the relevant income measure (e.g. expanded income). The add_quantile_table_row_variable() function creates the decile bins in two steps:decile_details=True
, which is the default option for difference and distribution tables, the original deciles are further broken down to include a group of filers with negative income and a group with 0 income.The problem arises if more than 10% of records have <=0 income since the bin sizes will no longer increase monotonically and pandas will throw an error when calling
pd.cut()
.This exact problem arose in #2444 when trying to run
recipe05
with the new CPS. Sincerecipe05
replaces expanded income with "market income" (i.e. does not include benefits received), the number of filers with <=0 income grew bigger than 10% of the records.A short term solution would include modifying
recipe05
. A more permanent solution would involve modifying the bin creation inutils.py
.The text was updated successfully, but these errors were encountered: