zfit binned

zfit binned#

There are two main ways of looking at “binned fits”

Either an analytic shape that could be fit unbinned but is fit to binned data because of the datasize (typical LHCb, Belle II,…)
stacking template histograms from simulation to provide the shape and fit to binned data (typical done in CMS, ATLAS, some LHCb,…)

Some templated fits with uniform binning, no analytic components and specific morphing and constraints fit into the HistFactory model, implemented in pyhf. These fits make a large portion of CMS and ATLAS analyses.

zfit can, in principle, reproduce them too. However, it’s comparably inefficient, a lot of code and slow. The main purpose is to support anything that is beyond HistFactory.

import matplotlib.pyplot as plt
import mplhep
import numpy as np
import zfit
import zfit.z.numpy as znp  # numpy-like backend interface

zfit.settings.set_seed(43)

{'zfit': np.int64(1678410587),
 'numpy': np.int64(1085174816),
 'backend': np.int64(685015468),
 'seed': 43}

Binned parts#

zfit introduces binned equivalents to unbinned components and transformations from one to the other. For example:

SumPDF -> BinnedSumPDF
Data -> BinnedData
UnbinnedNLL -> BinnedNLL

There are converters and new, histogram specific PDFs and methods.

From unbinned to binned#

Let’s start with an example, namely a simple, unbinned fit that we want to perform binned.

normal_np = np.random.normal(loc=2., scale=1.3, size=10000)

obs = zfit.Space("x", -10, 10)

mu = zfit.Parameter("mu", 1., -4, 6)
sigma = zfit.Parameter("sigma", 1., 0.1, 10)
model_nobin = zfit.pdf.Gauss(mu, sigma, obs)

data_nobin = zfit.Data(obs=obs, data=normal_np)

loss_nobin = zfit.loss.UnbinnedNLL(model_nobin, data_nobin)

minimizer = zfit.minimize.Minuit()

# make binned
nbins = 50
data = data_nobin.to_binned(nbins)
model = model_nobin.to_binned(data.space)

# we can create a binned NLL
loss = zfit.loss.BinnedNLL(model, data)

result = minimizer.minimize(loss)
print(result)

FitResult

 of
<BinnedNLL model=[<zfit.models.tobinned.BinnedFromUnbinnedPDF object at 0x755270602c90>] data=[<zfit._data.binneddatav1.BinnedData object at 0x7551d86df950>] constraints=[]> 
with
<Minuit Minuit, tol=0.001>

╒═════════╤═════════════╤══════════════════╤═════════╤══════════════════════════════╕
│  valid  │  converged  │  param at limit  │   edm   │   approx. fmin (full | opt.) │
╞═════════╪═════════════╪══════════════════╪═════════╪══════════════════════════════╡
│  

True

   │    True

     │      False

       │ 2.6e-05 │            94.37 |  4198.841 │
╘═════════╧═════════════╧══════════════════╧═════════╧══════════════════════════════╛

Parameters

name      value  (rounded)    at limit
------  ------------------  ----------
mu                 2.00581       False

sigma              1.30039       False

result.hesse(name="hesse")

{<zfit.Parameter 'mu' floating=True value=2.006>: {'error': np.float64(0.013055049948911726),
  'cl': 0.683,
  'weightcorr': <WeightCorr.FALSE: False>},
 <zfit.Parameter 'sigma' floating=True value=1.3>: {'error': np.float64(0.00926765607563971),
  'cl': 0.683,
  'weightcorr': <WeightCorr.FALSE: False>}}

result.errors(name="errors")

({<zfit.Parameter 'mu' floating=True value=2.006>: {'lower': -0.013144430719032691,
   'upper': 0.012966537992848583,
   'is_valid': True,
   'upper_valid': True,
   'lower_valid': True,
   'at_lower_limit': False,
   'at_upper_limit': False,
   'nfcn': 34,
   'original': ┌──────────┬───────────────────────┐
│          │          mu           │
├──────────┼───────────┬───────────┤
│  Error   │  -0.013   │   0.013   │
│  Valid   │   True    │   True    │
│ At Limit │   False   │   False   │
│ Max FCN  │   False   │   False   │
│ New Min  │   False   │   False   │
└──────────┴───────────┴───────────┘,
   'cl': 0.683},
  <zfit.Parameter 'sigma' floating=True value=1.3>: {'lower': -0.009233475555920441,
   'upper': 0.009301680009864553,
   'is_valid': True,
   'upper_valid': True,
   'lower_valid': True,
   'at_lower_limit': False,
   'at_upper_limit': False,
   'nfcn': 12,
   'original': ┌──────────┬───────────────────────┐
│          │         sigma         │
├──────────┼───────────┬───────────┤
│  Error   │  -0.009   │   0.009   │
│  Valid   │   True    │   True    │
│ At Limit │   False   │   False   │
│ Max FCN  │   False   │   False   │
│ New Min  │   False   │   False   │
└──────────┴───────────┴───────────┘,
   'cl': 0.683}},
 None)

print(result)

FitResult

 of
<BinnedNLL model=[<zfit.models.tobinned.BinnedFromUnbinnedPDF object at 0x755270602c90>] data=[<zfit._data.binneddatav1.BinnedData object at 0x7551d86df950>] constraints=[]> 
with
<Minuit Minuit, tol=0.001>

╒═════════╤═════════════╤══════════════════╤═════════╤══════════════════════════════╕
│  valid  │  converged  │  param at limit  │   edm   │   approx. fmin (full | opt.) │
╞═════════╪═════════════╪══════════════════╪═════════╪══════════════════════════════╡
│  

True

   │    True

     │      False

       │ 2.6e-05 │            94.37 |  4198.841 │
╘═════════╧═════════════╧══════════════════╧═════════╧══════════════════════════════╛

Parameters

name      value  (rounded)        hesse               errors    at limit
------  ------------------  -----------  -------------------  ----------
mu                 2.00581  +/-   0.013  -  0.013   +  0.013       False

sigma              1.30039  +/-  0.0093  - 0.0092   + 0.0093       False

Binned parts in detail#

to_binned creates a binned (and to_unbinned an unbinned) version of objects. It takes a binned Space, a binning or (as above), an integer (in which case a uniform binning is created).

This creates implicitly a new, binned space.

obs_binned_auto = data.space
print(obs_binned_auto)

<zfit Space obs=('x',), axes=(0,), limits=(array([[-10.]]), array([[10.]])), binned=True>

print(f"is_binned: {obs_binned_auto.is_binned}, binned obs binning: {obs_binned_auto.binning}")
print(f"is_binned: {obs.is_binned}, unbinned obs binning:{obs.binning}")

is_binned: True, binned obs binning: (RegularBinning(50, -10, 10, underflow=False, overflow=False, name='x'),)

is_binned: False, unbinned obs binning:None

Explicit conversion#

We can explicitly convert spaces, data and models to binned parts.

Either number of bins for uniform binning or explicit binning.

obs_binned = obs.with_binning(nbins)
print(obs_binned)

# or we can create binnings (same as boost-histogram/hist)
binning_regular = zfit.binned.RegularBinning(nbins, -10, 10, name='x')
binning_variable = zfit.binned.VariableBinning([-10, -6, -1, -0.1, 0.4, 3, 10], name='x')

<zfit Space obs=('x',), axes=None, limits=(array([[-10.]]), array([[10.]])), binned=True>

Since a binning contains all the information needed to create a Space, a binning can be used to define a space directly.

obs_binned_variable = zfit.Space(binning=binning_variable)
print(obs_binned_variable, obs_binned_variable.binning)

<zfit Space obs=('x',), axes=None, limits=(array([[-10.]]), array([[10.]])), binned=True>

(VariableBinning([-10, -6, -1, -0.1, 0.4, 3, 10], underflow=False, overflow=False, name='x'),)

Converting data, models#

data_nobin.to_binned(obs_binned_variable)

Variable([-10, -6, -1, -0.1, 0.4, 3, 10], underflow=False, overflow=False, name='x')

Weight() Σ=WeightedSum(value=10000, variance=10000)

model_nobin.to_binned(obs_binned_variable)

<zfit.models.tobinned.BinnedFromUnbinnedPDF at 0x7551d82c8860>

Compatibility with UHI#

zfit keeps compatibility with Universal Histogram Interface (UHI) and libraries that implement it (boost-histogram, hist).

BinnedData directly adheres to UHI (and has a to_hist attribute)
BinnedPDF has a to_binneddata and to_hist attribute

Where a BinnedData object is expected, a (named) UHI object is also possible. Same goes for the binning axes.

h = model.to_hist()
h_scaled = h * 10_000

Binneddata#

Binned data has counts, values and variances attributes, it has a binning (aliased with axes).

data.values()

<tf.Tensor: shape=(50,), dtype=float64, numpy=
array([0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00,
       0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00,
       0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00, 1.000e+00, 1.000e+00,
       3.000e+00, 8.000e+00, 1.300e+01, 3.100e+01, 9.700e+01, 1.460e+02,
       3.140e+02, 4.770e+02, 6.900e+02, 9.190e+02, 1.105e+03, 1.192e+03,
       1.204e+03, 1.053e+03, 9.410e+02, 6.810e+02, 4.950e+02, 2.960e+02,
       1.750e+02, 8.700e+01, 5.000e+01, 1.800e+01, 3.000e+00, 0.000e+00,
       0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00, 0.000e+00,
       0.000e+00, 0.000e+00])>

BinnedPDF#

A binned PDF has the same methods as the unbinned counterparts, namely pdf, integrate (and their ext_* parts) and sample that can respond to binned as well as unbinned data.

Additionally, there are two more methods, namely

counts returns the absolute counts as for a histogram. Equivalent to ext_pdf, ext_integrate, this only works if the PDF is extended.
rel_counts relative counts, like a histogram, but the sum is normalized to 1

Note on Counts vs Density#

Counts are the integrated density, i.e. they differ by a factor bin_width. For regular binning, this is “just” a constant factor, as it’s the same for all bins, but for Variable binning, this changes “the shape” of the histogram.

binned_sample = model.sample(n=1_000)

Plotting made easy#

This allows plotting to become a lot easier using mplhep, also for unbinned models.

plt.title("Counts plot")
mplhep.histplot(data, label="data")
mplhep.histplot(model.to_hist() * [data.nevents],
                label="model")  # scaling up since model is not extended, i.e. has no yield
plt.legend()

<matplotlib.legend.Legend at 0x7551d8563ce0>

../../_images/94301c862308daabae92e2f6263c4fd9ecf227c494418b926e3df38ef6197676.png

plt.title("Counts plot")
mplhep.histplot(binned_sample, label="sampled data")
mplhep.histplot(model.to_hist() * [binned_sample.nevents],
                label="model")  # scaling up since model is not extended, i.e. has no yield
plt.legend()

<matplotlib.legend.Legend at 0x7551d5e1b950>

../../_images/d0a4d08f2628b83794c199719bdf3b3ffc24a42da0bea57cdec0b7091d6f19a2.png

# or using unbinned data points, we can do a density plot
plt.title("Density plot")
mplhep.histplot(data.to_hist(), density=True, label="data")
x = znp.linspace(-10, 10, 200)
plt.plot(x, model.pdf(x), label="model")
plt.legend()

<matplotlib.legend.Legend at 0x7551d5dfecc0>

../../_images/f68daaec4b36adc469bcbe9d1438ad371c966a85ed1e1cc68dae0cd7ef81de02.png

Binned loss functions#

We used above the BinnedNLL, but zfit offers more, namely an extended version and a BinnedChi2 (or least-square).

print(zfit.loss.__all__)

['BaseLoss', 'BinnedChi2', 'BinnedNLL', 'ExtendedBinnedChi2', 'ExtendedBinnedNLL', 'ExtendedUnbinnedNLL', 'SimpleLoss', 'UnbinnedNLL', 'ZfitLoss']

Fitting using histograms#

There are a few new PDFs that are specific to histogram-like shapes, such as morphing interpolation and shape variations.

Most simple a HistogramPDF wraps a histogram and acts as a PDF.

By default, these histograms are extended automatically (which can be overruled using the extended argument)

histpdf = zfit.pdf.HistogramPDF(h_scaled)  # fixed yield
print(np.sum(histpdf.counts()))

10000.0

sig_yield = zfit.Parameter('sig_yield', 4_000, 0, 100_000)

histpdf = zfit.pdf.HistogramPDF(h, extended=sig_yield)
print(np.sum(histpdf.counts()))

4000.0000000000005

Modifiers#

We may want to add modifiers, i.e. scale each bin by a value. BinwiseScaleModifier offers this functionality.

Note however that these are just free parameters and not in any way constraint. This needs to be done manually.

histpdf.space.binning.size

(50,)

sig_pdf = zfit.pdf.BinwiseScaleModifier(histpdf,
                                        modifiers=True)  # or we could give a list of parameters matching each bin
modifiers = sig_pdf.params

# modifiers = {f'modifier_{i}': zfit.Parameter(f'modifier_{i}', 1, 0, 10) for i in range(histpdf.space.binning.size[0])}
# histpdf_scaled = zfit.pdf.BinwiseScaleModifier(histpdf, modifiers=modifiers)

modifiers

{'sysshape_0': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_0' floating=True value=1>,
 'sysshape_1': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_1' floating=True value=1>,
 'sysshape_2': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_2' floating=True value=1>,
 'sysshape_3': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_3' floating=True value=1>,
 'sysshape_4': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_4' floating=True value=1>,
 'sysshape_5': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_5' floating=True value=1>,
 'sysshape_6': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_6' floating=True value=1>,
 'sysshape_7': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_7' floating=True value=1>,
 'sysshape_8': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_8' floating=True value=1>,
 'sysshape_9': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_9' floating=True value=1>,
 'sysshape_10': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_10' floating=True value=1>,
 'sysshape_11': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_11' floating=True value=1>,
 'sysshape_12': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_12' floating=True value=1>,
 'sysshape_13': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_13' floating=True value=1>,
 'sysshape_14': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_14' floating=True value=1>,
 'sysshape_15': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_15' floating=True value=1>,
 'sysshape_16': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_16' floating=True value=1>,
 'sysshape_17': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_17' floating=True value=1>,
 'sysshape_18': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_18' floating=True value=1>,
 'sysshape_19': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_19' floating=True value=1>,
 'sysshape_20': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_20' floating=True value=1>,
 'sysshape_21': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_21' floating=True value=1>,
 'sysshape_22': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_22' floating=True value=1>,
 'sysshape_23': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_23' floating=True value=1>,
 'sysshape_24': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_24' floating=True value=1>,
 'sysshape_25': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_25' floating=True value=1>,
 'sysshape_26': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_26' floating=True value=1>,
 'sysshape_27': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_27' floating=True value=1>,
 'sysshape_28': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_28' floating=True value=1>,
 'sysshape_29': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_29' floating=True value=1>,
 'sysshape_30': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_30' floating=True value=1>,
 'sysshape_31': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_31' floating=True value=1>,
 'sysshape_32': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_32' floating=True value=1>,
 'sysshape_33': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_33' floating=True value=1>,
 'sysshape_34': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_34' floating=True value=1>,
 'sysshape_35': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_35' floating=True value=1>,
 'sysshape_36': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_36' floating=True value=1>,
 'sysshape_37': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_37' floating=True value=1>,
 'sysshape_38': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_38' floating=True value=1>,
 'sysshape_39': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_39' floating=True value=1>,
 'sysshape_40': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_40' floating=True value=1>,
 'sysshape_41': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_41' floating=True value=1>,
 'sysshape_42': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_42' floating=True value=1>,
 'sysshape_43': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_43' floating=True value=1>,
 'sysshape_44': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_44' floating=True value=1>,
 'sysshape_45': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_45' floating=True value=1>,
 'sysshape_46': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_46' floating=True value=1>,
 'sysshape_47': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_47' floating=True value=1>,
 'sysshape_48': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_48' floating=True value=1>,
 'sysshape_49': <zfit.Parameter 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_49' floating=True value=1>}

sig_pdf.get_yield()

<zfit.ComposedParameter 'AUTO_binwise_modifier_2' params=[('sysshape_0', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_0'), ('sysshape_1', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_1'), ('sysshape_2', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_2'), ('sysshape_3', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_3'), ('sysshape_4', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_4'), ('sysshape_5', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_5'), ('sysshape_6', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_6'), ('sysshape_7', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_7'), ('sysshape_8', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_8'), ('sysshape_9', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_9'), ('sysshape_10', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_10'), ('sysshape_11', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_11'), ('sysshape_12', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_12'), ('sysshape_13', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_13'), ('sysshape_14', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_14'), ('sysshape_15', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_15'), ('sysshape_16', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_16'), ('sysshape_17', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_17'), ('sysshape_18', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_18'), ('sysshape_19', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_19'), ('sysshape_20', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_20'), ('sysshape_21', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_21'), ('sysshape_22', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_22'), ('sysshape_23', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_23'), ('sysshape_24', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_24'), ('sysshape_25', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_25'), ('sysshape_26', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_26'), ('sysshape_27', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_27'), ('sysshape_28', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_28'), ('sysshape_29', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_29'), ('sysshape_30', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_30'), ('sysshape_31', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_31'), ('sysshape_32', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_32'), ('sysshape_33', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_33'), ('sysshape_34', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_34'), ('sysshape_35', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_35'), ('sysshape_36', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_36'), ('sysshape_37', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_37'), ('sysshape_38', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_38'), ('sysshape_39', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_39'), ('sysshape_40', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_40'), ('sysshape_41', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_41'), ('sysshape_42', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_42'), ('sysshape_43', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_43'), ('sysshape_44', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_44'), ('sysshape_45', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_45'), ('sysshape_46', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_46'), ('sysshape_47', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_47'), ('sysshape_48', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_48'), ('sysshape_49', 'auto_sysshape_<zfit.models.histmodifier.BinwiseScaleModifier object at 0x7551d5c20c50>_49'), ('DUMMPY_PARAM_1', 'sig_yield')] value=4000>

Morphing#

Let’s create a background from simulation. Let’s assume, we have a parameter in the simulation that we’re unsure about.

A common used technique is to use “morphing”: creating multiple templates and interpolating between them. Typically, they are created at +1 and -1 sigma of the nuisance parameter (however, zfit allows arbitrary values and as many as wanted)

bkg_hist = zfit.Data(np.random.exponential(scale=20, size=100_000) - 10, obs=obs_binned)

# creating templates, different ways of going about it
# 1. create unbinned and convert to binned
bkg_m1_unbinned = zfit.Data(obs=obs, data=np.random.exponential(scale=35, size=100_000) - 10)
bkg_hist_m1 = bkg_m1_unbinned.to_binned(obs_binned)

# 2. directly create binned by using the binned observables
bkg_hist_m05 = zfit.Data(obs=obs_binned, data=np.random.exponential(scale=26, size=100_000) - 10)

# 3. use the `from_numpy` method that has more specific options than just `Data`
bkg_hist_p1 = zfit.data.from_numpy(obs=obs_binned, array=np.random.exponential(scale=17, size=100_000) - 10)

# put them into a dict that maps the modifier value to the histogram
bkg_hists = {-1: bkg_hist_m1, -0.5: bkg_hist_m05, 0: bkg_hist, 1: bkg_hist_p1}
bkg_histpdfs = {k: zfit.pdf.HistogramPDF(v) for k, v in bkg_hists.items()}
mplhep.histplot(list(bkg_hists.values()), label=list(bkg_hists.keys()))
plt.legend();

../../_images/9247f76ad955f838aedb2f5d80e7cfe4f917be4a0c378e7ca9ad4bbf8e92c1cf.png

alpha = zfit.Parameter("alpha", 0, -3, 3)
bkg_yield = zfit.Parameter("bkg_yield", 15_000)

bkg_pdf = zfit.pdf.SplineMorphingPDF(alpha, bkg_histpdfs, extended=bkg_yield)

with alpha.set_value(-0.6):  # we can change this value to play around
    mplhep.histplot(bkg_pdf.to_hist())

../../_images/3be2854546bf8583fa73193e8f78cd625ba0b89c8efbc8c3cd8f56ab4db9ad50.png

# bkg_pdf = zfit.pdf.HistogramPDF(bkg_hist, extended=bkg_yield)  # we don't use the spline for simplicity

model = zfit.pdf.BinnedSumPDF([sig_pdf, bkg_pdf])
model.to_hist()

Regular(50, -10, 10, underflow=False, overflow=False, name='x')

Weight() Σ=WeightedSum(value=19000, variance=0)

mods_signal = {m: np.random.normal(1.0, scale=0.14) for m in modifiers.values()}
mods_bkg = {alpha: 0.1}

bkghist = bkg_pdf.sample(n=10_000, params=mods_bkg).to_hist()
sighist = sig_pdf.sample(1000, params=mods_signal).to_hist()
data = bkghist + sighist
data