Metadata and logging

Metadata and logging#

Papylio contains several ways to keep track of your actions.

log file#

First, for every file a corresponding .log file is created that keeps track of actions performed on the File class. These actions include setting of attributes and running of methods, where the used method arguments are also reported.

For example after running the following commands:

[2]:

file.test = 'test'

file.find_coordinates(channels=['donor', 'acceptor'],
                      method='sum_channels',
                      projection_image=dict(projection_type='average',
                                            frame_range=(0, 20),
                                            illumination=0),
                      peak_finding=dict(method='local-maximum-auto',
                                        fraction_difference=0.35,
                                        filter_neighbourhood_size_min=10,
                                        filter_neighbourhood_size_max=3),
                      coordinate_optimization=dict(coordinates_within_margin=dict(margin=8),
                                                   coordinates_after_gaussian_fit=dict(gaussian_width=3)
                                                   )
                    )

file.extract_traces(mask_size=1.01, background_correction=(-300, -100), alpha_correction=0.05, gamma_correction=1.5)

 Finding molecules in File(ssHJ1\ssHJ1 TIRF 561 0001)
   900 molecules found
  Extracting traces in File(ssHJ1\ssHJ1 TIRF 561 0001)

ssHJ1 TIRF 561 0001: 100%|██████████| 400/400 [00:01<00:00, 255.79it/s]

The log file contains

[3]:

with file._log_filepath.open("r") as f:
    contents = f.read()
    print(contents)

2025-11-05 15:04:13 [INFO]: Initialized File(ssHJ1\ssHJ1 TIRF 561 0001) with Papylio v0.8+g67689f2bf.d20251105
2025-11-05 15:04:16 [INFO]: Set attribute mapping = <matchpoint.core.MatchPoint object at 0x0000020D1FDADA90>
2025-11-05 15:04:16 [INFO]: Set attribute is_mapping_file = False
2025-11-05 15:04:16 [INFO]: Set attribute test = 'test'
2025-11-05 15:04:16 [INFO]: Called find_coordinates(channels=['donor', 'acceptor'], method='sum_channels', projection_image={'projection_type': 'average', 'frame_range': (0, 20), 'illumination': 0}, peak_finding={'method': 'local-maximum-auto', 'fraction_difference': 0.35, 'filter_neighbourhood_size_min': 10, 'filter_neighbourhood_size_max': 3}, coordinate_optimization={'coordinates_within_margin': {'margin': 8}, 'coordinates_after_gaussian_fit': {'gaussian_width': 3}})
2025-11-05 15:04:18 [INFO]: Called extract_traces(mask_size=1.01, background_correction=(-300, -100), alpha_correction=0.05, gamma_correction=1.5)

Logging can be turned off for all files when initializing the experiment:

[4]:

pp.Experiment(experiment_path, perform_logging=False)

Import files: 100%|██████████| 21/21 [00:02<00:00,  8.19it/s]


File(Bead slide\Bead slide) used as mapping

Initialize experiment:
C:\Users\IVOSEV~1\AppData\Local\Temp\tmpw5tgg_pu

[4]:

Experiment(tmpw5tgg_pu)

or for specific files using

[5]:

file.perform_logging = False

NetCDF file attributes#

Additionally, each of the produced dataarrays within the datasets contain metadata in the dataarray attributes. These include the version attribute containing the version of Papylio used to create the array, the datetime attribute reporting the date and time at which the array was created, and the configuration attribute containing the used function arguments. There may be additional attributes, such as units.

[6]:

file.coordinates

[6]:

<xarray.DataArray 'coordinates' (molecule: 900, channel: 2, dimension: 2)>
array([[[ 93.40787629, 151.45196699],
        [350.47893884, 151.18019534]],

       [[227.3520365 , 200.40436677],
        [483.68108776, 199.92067337]],

       [[240.28437069, 337.23512685],
        [496.44103552, 336.43233556]],

       ...,

       [[134.08443341,  85.56563263],
        [391.02887905,  85.50130553]],

       [[202.57793233, 458.36333123],
        [458.91017616, 457.23041479]],

       [[ 18.40207882, 125.63105276],
        [275.9976018 , 125.48245048]]])
Coordinates:
    molecule_in_file  (molecule) int32 0 1 2 3 4 5 6 ... 894 895 896 897 898 899
    file              (molecule) |S25 b'ssHJ1\\ssHJ1 TIRF 561 0001' ... b'ssH...
  * channel           (channel) int64 0 1
  * dimension         (dimension) |S1 b'x' b'y'
Dimensions without coordinates: molecule
Attributes:
    version:            0.8+g67689f2bf.d20251105
    stage_coordinates:  [-11409.2    356.4]
    datetime:           2025-11-05 15:04:18
    pixel_size_unit:    µm
    pixel_size:         [0.125 0.125]
    configuration:      {"channels": ["donor", "acceptor"], "illumination": 0...
    units:              pixel

The configuration attribute can be used to reapply the analysis.

As it is stored as a json string, first it needs to be transformed to a dictionary and then it can be directly used in the method:

[7]:

configuration = file.coordinates.attrs['configuration']

import json
configuration = json.loads(configuration)

file.find_coordinates(**configuration)

 Finding molecules in File(ssHJ1\ssHJ1 TIRF 561 0001)
   900 molecules found
Contents