Is it possible to define a data transformation for specific groups in a tsibble when applying a forecasting model like ETS or ARIMA?

Question

I have a tsibble made of a date column (the index, with monthly frequency), a geo column (the key, representing some European states), and a value column (the variable I need to forecast).

I am fitting ETS and ARIMA to this dataset, and then forecast with a time horizon of 3 months, like so:

fit <- data |>
  model(
    ets = decomposition_model(
      STL(value),
      ETS(season_adjust),
      SNAIVE(season_year)
    ),
    arima = ARIMA(value)
  )

fc <- fit |> forecast(h = 3)

It works as expected. Now, I would like to apply some data transformations on some of the States for which forecasts are not satisfactory. For instance, I have seen that for HU if I specify a log-transform inside the model, the forecasts are a bit better (at least the scale of the values). Something like this, but only for specific states:

fit <- data |>
  model(
    ets = decomposition_model(
      STL(log(value)+1), # THIS!
      ETS(season_adjust),
      SNAIVE(season_year)
    ),
    arima = ARIMA(log(value)+1) # THIS!
  )

Is there a way to include such information, maybe inside the tsibble so that when fitting it is taken into account?

This is the datset (a small part since it is very large, but should be enough):

structure(list(date = structure(c(13879, 13910, 13939, 13970, 
14000, 14031, 14061, 14092, 14123, 14153, 14184, 14214, 14245, 
14276, 14304, 14335, 14365, 14396, 14426, 14457, 14488, 14518, 
14549, 14579, 14610, 14641, 14669, 14700, 14730, 14761, 14791, 
14822, 14853, 14883, 14914, 14944, 14975, 14975, 13879, 13910, 
13939, 13970, 14000, 14031, 14061, 14092, 14123, 14153, 14184, 
14214, 14245, 14276, 14304, 14335, 14365, 14396, 14426, 14457, 
14488, 14518, 14549, 14579, 14610, 14641, 14669, 14700, 14730, 
14761, 14791, 14822, 14853, 14883, 14914, 14944, 14975, 13879, 
13910, 13939, 13970, 14000, 14031, 14061, 14092, 14123, 14153, 
14184, 14214, 14245, 14276, 14304, 14335, 14365, 14396, 14426, 
14457, 14488, 14518, 14549, 14579, 14610, 14641, 14669, 14700, 
14730, 14761, 14791, 14822, 14853, 14883, 14914, 14944, 14975, 
13879, 13910, 13939, 13970, 14000, 14031, 14061, 14092, 14123, 
14153, 14184, 14214, 14245, 14276, 14304, 14335, 14365, 14396, 
14426, 14457, 14488, 14518, 14549, 14579, 14610, 14641, 14669, 
14700, 14730, 14761, 14791, 14822, 14853, 14883, 14914, 14944, 
14975, 13879, 13910, 13939, 13970, 14000, 14031, 14061, 14092, 
14123, 14153, 14184, 14214, 14245, 14276, 14304, 14335, 14365, 
14396, 14426, 14457, 14488, 14518, 14549, 14579, 14610, 14641, 
14669, 14700, 14730, 14761, 14791, 14822, 14853, 14883, 14914, 
14944, 14975, 13879, 13910, 13939, 13970, 14000, 14031, 14061, 
14092, 14123, 14153, 14184, 14214, 14245, 14276, 14304, 14335, 
14365, 14396, 14426, 14457, 14488, 14518, 14549, 14579, 14610, 
14641, 14669, 14700, 14730, 14761, 14791, 14822, 14853, 14883, 
14914, 14944, 14975, 14975, 14610, 14641, 14669, 14700, 14730, 
14761, 14791, 14822, 14853, 14883, 14914, 14944, 14975, 14245, 
14276, 14304, 14335, 14365, 14396, 14426, 14457, 14488, 14518, 
14549, 14579, 14610, 14641, 14669, 14700, 14730, 14761, 14791, 
14822, 14853, 14883, 14914, 14944, 14975, 13879, 13910, 13939, 
13970, 14000, 14031, 14061, 14092, 14123, 14153, 14184, 14214, 
14245, 14276, 14304, 14335, 14365, 14396, 14426, 14457, 14488, 
14518, 14549, 14579, 14610, 14641, 14669, 14700, 14730, 14761, 
14791, 14822, 14853, 14883, 14914, 14944, 14975, 13879, 13910, 
13939, 13970, 14000, 14031, 14061, 14092, 14123, 14153, 14184, 
14214, 14245, 14276, 14304, 14335, 14365, 14396, 14426, 14457, 
14488, 14518, 14549, 14579, 14610, 14641, 14669, 14700, 14730, 
14761, 14791, 14822, 14853, 14883, 14914, 14944, 14975, 14245, 
14276, 14304, 14335, 14365, 14396, 14426, 14457, 14488, 14518, 
14549, 14579, 14610, 14641, 14669, 14700, 14730, 14761, 14791, 
14822, 14853, 14883, 14914, 14944, 14975, 14669, 14700, 14730, 
14761, 14791, 14822, 14853, 14883, 14914, 14944, 14975, 13879, 
13910, 13939, 13970, 14000, 14031, 14061, 14092, 14123, 14153, 
14184, 14214, 14245, 14276, 14304, 14335, 14365, 14396, 14426, 
14457, 14488, 14518, 14549, 14579, 14610, 14641, 14669, 14700, 
14730, 14761, 14791, 14822, 14853, 14883, 14914, 14944, 14975, 
13879, 13910, 13939, 13970, 14000, 14031, 14061, 14092, 14123, 
14153, 14184, 14214, 14245, 14276, 14304, 14335, 14365, 14396, 
14426, 14457, 14488, 14518, 14549, 14579, 14610, 14641, 14669, 
14700, 14730, 14761, 14791, 14822, 14853, 14883, 14914, 14944, 
14975, 13879, 13910, 13939, 13970, 14000, 14031, 14061, 14092, 
14123, 14153, 14184, 14214, 14245, 14276, 14304, 14335, 14365, 
14396, 14426, 14457, 14488, 14518, 14549, 14579, 14610, 14641, 
14669, 14700, 14730, 14761, 14791, 14822, 14853, 14883, 14914, 
14944, 14975, 13879, 13910, 13939, 13970, 14000, 14031, 14061, 
14092, 14123, 14153, 14184, 14214, 14245, 14276, 14304, 14335, 
14365, 14396, 14426, 14457, 14488, 14518, 14549, 14579, 14610, 
14641, 14669, 14700, 14730, 14761, 14791, 14822, 14853, 14883, 
14914, 14944, 14975, 13879, 13910, 13939, 13970, 14000, 14031, 
14061, 14092, 14123, 14153, 14184, 14214, 14245, 14276, 14304, 
14335, 14365, 14396, 14426, 14457, 14488, 14518, 14549, 14579, 
14610, 14641, 14669, 14700, 14730, 14761, 14791, 14822, 14853, 
14883, 14914, 14944, 14975, 14975, 13879, 13910, 13939, 13970, 
14000, 14031, 14061, 14092, 14123, 14153, 14184, 14214, 14245, 
14276, 14304, 14335, 14365, 14396, 14426, 14457, 14488, 14518, 
14549, 14579, 14610, 14641, 14669, 14700, 14730, 14761, 14791, 
14822, 14853, 14883, 14914, 14944, 14975, 13879, 13910, 13939, 
13970, 14000, 14031, 14061, 14092, 14123, 14153, 14184, 14214, 
14245, 14276, 14304, 14335, 14365, 14396, 14426, 14457, 14488, 
14518, 14549, 14579, 14610, 14641, 14669, 14700, 14730, 14761, 
14791, 14822, 14853, 14883, 14914, 14944, 14975, 14610, 14641, 
14669, 14700, 14730, 14761, 14791, 14822, 14853, 14883, 14914, 
14944, 14975), class = c("yearmonth", "vctrs_vctr")), geo = c("BE", 
"BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", 
"BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", 
"BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", "BE", 
"BE", "BE", "BE", "BG", "CY", "CY", "CY", "CY", "CY", "CY", "CY", 
"CY", "CY", "CY", "CY", "CY", "CY", "CY", "CY", "CY", "CY", "CY", 
"CY", "CY", "CY", "CY", "CY", "CY", "CY", "CY", "CY", "CY", "CY", 
"CY", "CY", "CY", "CY", "CY", "CY", "CY", "CY", "CZ", "CZ", "CZ", 
"CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", 
"CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", 
"CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", "CZ", 
"CZ", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", 
"DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", 
"DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", "DE", 
"DE", "DE", "DE", "DE", "DE", "DK", "DK", "DK", "DK", "DK", "DK", 
"DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", 
"DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", 
"DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", "DK", "EE", "EE", 
"EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", 
"EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", 
"EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", "EE", 
"EE", "EE", "EL", "ES", "ES", "ES", "ES", "ES", "ES", "ES", "ES", 
"ES", "ES", "ES", "ES", "ES", "FR", "FR", "FR", "FR", "FR", "FR", 
"FR", "FR", "FR", "FR", "FR", "FR", "FR", "FR", "FR", "FR", "FR", 
"FR", "FR", "FR", "FR", "FR", "FR", "FR", "FR", "IE", "IE", "IE", 
"IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", 
"IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", 
"IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", "IE", 
"IE", "IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", 
"IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", 
"IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", "IT", 
"IT", "IT", "IT", "IT", "IT", "LT", "LT", "LT", "LT", "LT", "LT", 
"LT", "LT", "LT", "LT", "LT", "LT", "LT", "LT", "LT", "LT", "LT", 
"LT", "LT", "LT", "LT", "LT", "LT", "LT", "LT", "LU", "LU", "LU", 
"LU", "LU", "LU", "LU", "LU", "LU", "LU", "LU", "LV", "LV", "LV", 
"LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", 
"LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", 
"LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", "LV", 
"LV", "MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", 
"MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", 
"MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", "MT", 
"MT", "MT", "MT", "MT", "MT", "NL", "NL", "NL", "NL", "NL", "NL", 
"NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", 
"NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", 
"NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", "NL", "PL", "PL", 
"PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", 
"PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", 
"PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", "PL", 
"PL", "PL", "PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", 
"PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", 
"PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", "PT", 
"PT", "PT", "PT", "PT", "PT", "PT", "RO", "SE", "SE", "SE", "SE", 
"SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", 
"SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", 
"SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", "SE", 
"SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", 
"SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", 
"SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", "SI", 
"SI", "SI", "SI", "SI", "SK", "SK", "SK", "SK", "SK", "SK", "SK", 
"SK", "SK", "SK", "SK", "SK", "SK"), value = c(1060L, 845L, 735L, 
725L, 805L, 890L, 935L, 940L, 1035L, 1230L, 1015L, 885L, 1270L, 
955L, 1010L, 990L, 985L, 1045L, 1210L, 1275L, 1550L, 1910L, 2050L, 
2355L, 1240L, 2480L, 1520L, 1145L, 1255L, 1320L, 1550L, 1965L, 
2245L, 2450L, 2225L, 2160L, 1870L, 100L, 245L, 440L, 425L, 310L, 
315L, 275L, 435L, 220L, 325L, 230L, 335L, 365L, 305L, 370L, 220L, 
210L, 270L, 240L, 245L, 175L, 270L, 315L, 335L, 245L, 215L, 210L, 
190L, 175L, 210L, 230L, 205L, 165L, 190L, 210L, 350L, 485L, 185L, 
155L, 150L, 135L, 100L, 60L, 55L, 50L, 60L, 70L, 110L, 65L, 45L, 
80L, 65L, 50L, 50L, 60L, 45L, 45L, 35L, 40L, 50L, 45L, 55L, 30L, 
35L, 55L, 40L, 35L, 35L, 20L, 35L, 25L, 20L, 20L, 40L, 30L, 2395L, 
1815L, 1535L, 1690L, 1595L, 1670L, 1790L, 1660L, 1965L, 1945L, 
1730L, 1540L, 2435L, 2035L, 2075L, 1950L, 1885L, 2000L, 2605L, 
2525L, 2690L, 2685L, 2515L, 2170L, 2715L, 2435L, 2705L, 2455L, 
2410L, 2890L, 3375L, 4085L, 4765L, 4975L, 4740L, 3695L, 3895L, 
235L, 165L, 130L, 160L, 155L, 145L, 130L, 210L, 250L, 245L, 275L, 
245L, 335L, 260L, 340L, 215L, 300L, 205L, 225L, 315L, 375L, 355L, 
410L, 385L, 405L, 420L, 390L, 370L, 390L, 320L, 370L, 535L, 455L, 
540L, 425L, 440L, 360L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
5L, 0L, 0L, 5L, 5L, 0L, 0L, 0L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 0L, 
5L, 5L, 5L, 0L, 5L, 5L, 0L, 0L, 0L, 0L, 5L, 605L, 165L, 200L, 
205L, 185L, 170L, 190L, 245L, 245L, 265L, 220L, 270L, 185L, 205L, 
2905L, 3755L, 3640L, 3490L, 2815L, 3475L, 3550L, 3200L, 3480L, 
4135L, 3720L, 3905L, 3325L, 4350L, 4600L, 3835L, 3720L, 3800L, 
3755L, 3620L, 3830L, 4655L, 3910L, 4625L, 4150L, 310L, 290L, 
295L, 295L, 295L, 320L, 330L, 325L, 360L, 355L, 320L, 295L, 250L, 
270L, 270L, 235L, 225L, 240L, 205L, 225L, 195L, 205L, 175L, 160L, 
190L, 175L, 165L, 160L, 180L, 155L, 160L, 170L, 150L, 140L, 155L, 
120L, 135L, 1170L, 1105L, 1800L, 1780L, 2850L, 2680L, 3870L, 
3785L, 3395L, 3260L, 2740L, 1705L, 1715L, 1375L, 1470L, 2280L, 
2395L, 1240L, 1415L, 590L, 1080L, 1305L, 1850L, 655L, 1000L, 
675L, 1030L, 730L, 940L, 975L, 815L, 515L, 810L, 865L, 1020L, 
630L, 815L, 15L, 15L, 10L, 10L, 20L, 10L, 5L, 20L, 50L, 25L, 
15L, 20L, 10L, 25L, 15L, 5L, 35L, 20L, 20L, 35L, 30L, 55L, 75L, 
50L, 10L, 50L, 45L, 45L, 40L, 40L, 50L, 90L, 90L, 105L, 95L, 
105L, 0L, 0L, 0L, 0L, 0L, 5L, 0L, 10L, 0L, 20L, 10L, 5L, 0L, 
0L, 0L, 0L, 0L, 0L, 10L, 5L, 5L, 20L, 5L, 0L, 0L, 5L, 0L, 10L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 220L, 150L, 110L, 200L, 190L, 
230L, 365L, 270L, 260L, 190L, 205L, 215L, 220L, 225L, 225L, 200L, 
180L, 155L, 145L, 205L, 145L, 160L, 280L, 250L, 25L, 5L, 0L, 
10L, 10L, 0L, 40L, 10L, 10L, 10L, 10L, 15L, 10L, 905L, 860L, 
885L, 925L, 1010L, 1165L, 1270L, 1285L, 1445L, 1435L, 1105L, 
1080L, 1170L, 1145L, 1300L, 1230L, 1020L, 1215L, 1170L, 1030L, 
1325L, 1570L, 1465L, 1230L, 1300L, 1155L, 1345L, 950L, 750L, 
890L, 830L, 1060L, 1240L, 1310L, 1415L, 1050L, 1045L, 360L, 635L, 
510L, 565L, 460L, 480L, 490L, 585L, 950L, 710L, 645L, 810L, 310L, 
290L, 415L, 405L, 445L, 1250L, 1490L, 1970L, 920L, 805L, 620L, 
745L, 310L, 330L, 315L, 285L, 285L, 290L, 280L, 460L, 505L, 415L, 
375L, 485L, 125L, 5L, 15L, 10L, 15L, 10L, 35L, 5L, 20L, 10L, 
15L, 10L, 5L, 15L, 10L, 15L, 5L, 10L, 25L, 10L, 5L, 10L, 15L, 
5L, 10L, 10L, 20L, 5L, 10L, 15L, 15L, 15L, 20L, 15L, 10L, 5L, 
15L, 10L, 45L, 3180L, 2205L, 1635L, 1830L, 1740L, 1630L, 1895L, 
1925L, 2140L, 2190L, 1840L, 2060L, 1860L, 1595L, 1645L, 1500L, 
1545L, 1835L, 2020L, 2360L, 2150L, 2280L, 2355L, 2465L, 2225L, 
2325L, 3565L, 2150L, 1870L, 1910L, 1895L, 2735L, 3880L, 4190L, 
2830L, 2210L, 1960L, 15L, 15L, 10L, 15L, 45L, 10L, 15L, 10L, 
25L, 25L, 35L, 20L, 30L, 10L, 15L, 10L, 10L, 5L, 30L, 10L, 10L, 
0L, 10L, 30L, 10L, 5L, 10L, 0L, 20L, 15L, 10L, 0L, 20L, 25L, 
40L, 40L, 30L, 25L, 10L, 25L, 40L, 35L, 40L, 20L, 30L, 35L, 25L, 
15L, 15L, 15L)), class = c("tbl_ts", "tbl_df", "tbl", "data.frame"
), row.names = c(NA, -645L), key = structure(list(geo = c("BE", 
"BG", "CY", "CZ", "DE", "DK", "EE", "EL", "ES", "FR", "IE", "IT", 
"LT", "LU", "LV", "MT", "NL", "PL", "PT", "RO", "SE", "SI", "SK"
), .rows = structure(list(1:37, 38L, 39:75, 76:112, 113:149, 
    150:186, 187:223, 224L, 225:237, 238:262, 263:299, 300:336, 
    337:361, 362:372, 373:409, 410:446, 447:483, 484:520, 521:557, 
    558L, 559:595, 596:632, 633:645), ptype = integer(0), class = c("vctrs_list_of", 
"vctrs_vctr", "list"))), class = c("tbl_df", "tbl", "data.frame"
), row.names = c(NA, -23L), .drop = TRUE), index = structure("date", ordered = TRUE), index2 = "date", interval = structure(list(
    year = 0, quarter = 0, month = 1, week = 0, day = 0, hour = 0, 
    minute = 0, second = 0, millisecond = 0, microsecond = 0, 
    nanosecond = 0, unit = 0), .regular = TRUE, class = c("interval", 
"vctrs_rcrd", "vctrs_vctr")))

Collectives™ on Stack Overflow

Is it possible to define a data transformation for specific groups in a tsibble when applying a forecasting model like ETS or ARIMA?

0

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

0

Know someone who can answer? Share a link to this question via email, Twitter, or Facebook.

Your Answer

Sign up or log in

Post as a guest