Skip to main content

Data Handling

This tutorial describes how to start working with the Proximity Network Assay (PNA) output data format. The primary data output file of pixelator is the PXL file. Here, you will learn how to read and interact with it. We will go through the different components of a single sample PXL file and also how to aggregate results from multiple samples. You can read more about the PXL file format here.

After completing this tutorial, you will be able to:

  • Load PXL files in Python and access the multi-modal data contents including protein counts, metadata and spatial scores.
  • Understand and work with spatial scores, i.e. protein clustering and colocalization (co-clustering) patterns.
  • Directly access the spatial graph structure through the edge list.
  • Aggregate multiple PXL files into an integrated data object with sample identities.

Setup​

To start with, we need to load some packages and functions that we will need.

from pixelator import read_pna as read
from pathlib import Path
import os

And define the directory where example data will be stored when you download it.

DATA_DIR = Path("./pna-analysis")

Loading data​

We begin by locating the output from the pixelator pipeline. The input for downstream analysis is the PXL file contained in the layout directory.

from pixelator.pna.pixeldataset.download import DownloadableDatasets

DATA_DIR.mkdir(parents=True, exist_ok=True)

DownloadableDatasets.download_dataset(
"pna062-unstim-pbmcs",
output_path=DATA_DIR / "PNA062_unstim_PBMCs_1000cells_S02_S2.layout.pxl",
)
pg_data = read(DATA_DIR / "PNA062_unstim_PBMCs_1000cells_S02_S2.layout.pxl")

Component meta data​

The object contains antibody count and meta data for each connected component in the graph. Each graph component corresponds to a cell. This table contains information that can be useful in quality control, such as how many protein molecules were detected (n_umi), the sequencing depth (reads_in_component), and the graph connectivity of each component (average_k_core).

pg_data.adata().obs
n_umi1n_umi2n_edgesreads_in_componentn_antibodiesn_umiisotype_fractionintracellular_fractiontau_typetau...k_core_1k_core_2k_core_3k_core_4k_core_5k_core_6k_core_7svd_var_expl_s1svd_var_expl_s2svd_var_expl_s3
component
f1b52a4758932fc7133551831193749246004133316660.0003160.0normal0.974145...5773.03009.05429.08402.09053.00.00.00.2844170.1568090.089341
eca4983191f3647f1731123376105495273174139406870.0002210.0normal0.975355...7919.05180.09614.017974.00.00.00.00.2535570.1929810.146433
f98240c66b2e49fc118271493478611225033158267610.0017940.0normal0.946812...4706.03152.03553.08152.07198.00.00.00.2395870.1878860.120709
b5f8b192a0cc2603214502811197857296160158495610.0004040.0normal0.949583...12780.012920.019498.04363.00.00.00.00.3493260.2372030.116332
7994e60e303c6dbb1576720905126994357256144366720.0002180.0normal0.952086...5937.02389.03245.07657.014342.03102.00.00.2922640.1730780.140349
..................................................................
d3fd6df73f0c660a81781007048412132220126182480.0002190.0normal0.966714...3556.02407.03715.07959.0611.00.00.00.2651770.2100680.106871
38663b1fa7dd363a633178013423995228100141320.0002120.0normal0.987860...2910.02406.03488.05328.00.00.00.00.3027130.2648910.151157
9791359e16ac85ae4146490924284653038590550.0003310.0normal0.991820...1751.01253.01412.04349.0290.00.00.00.2879430.2018340.128967
072963fbd47582c3128231620971683204981137290320.0001030.0normal0.963505...5543.04191.07566.011732.00.00.00.00.3345280.2126750.144832
b5e265cb02443e7b46455854390949748589104990.0000950.0normal0.992041...1603.0690.0879.01371.02480.03476.00.00.3268710.1158630.095980

1083 rows × 25 columns

The antibody count table is also present in the data object.

pg_data.adata().to_df()
marker_idHLA-ABCB2MCD11bCD11cCD18CD82CD8TCRabHLA-DRCD45...CX3CR1CD326CD209CD34CD369CD54CD71CD47CD117CD314
component
f1b52a4758932fc71643340213118376816112141...0000316216441
eca4983191f3647f1719358701281781432223939...4000525322134
f98240c66b2e49fc1029192213712445286291101575...3157109871301334
b5f8b192a0cc260319844153742875473073564236702454...27631229245141336192
7994e60e303c6dbb206543032697404240163132096...0012462119441
..................................................................
d3fd6df73f0c660a152629312271451141171094...30003744111444
38663b1fa7dd363a1260205443234900126...00060334810
9791359e16ac85ae7989571038220114...11160007510
072963fbd47582c318614314603131318345531641...80001991122231
b5e265cb02443e7b3854600021201236...100141117000

1083 rows × 158 columns

Spatial data​

In addition to count data corresponding to protein abundance, proximity network assay also generates spatial data which is used to find spatial structures of proteins across cells and to visualize individual cells. The PXL file comes with proximity scores, describing the degree of spatial clustering of each protein, and the colocalization (co-clustering) of each protein-pair on each cell. If you want to read more about spatial metrics in proximity network assay data, LINK_TO_PNA_PROXIMITY_EXPLANATION.

The proximity scores can be retrieved through proximity() function.

pg_data.proximity().to_df()
marker_1marker_2join_countjoin_count_expected_meanjoin_count_expected_sdjoin_count_zjoin_count_pcomponentsamplemarker_1_countmarker_1_freqmarker_2_countmarker_2_freqmin_countlog2_ratio
0CD33CD3300.250.519810-0.2500000.40129491984005700471b2PNA062_unstim_PBMCs_1000cells_S02_S2340.003074340.003074340.000000
1CD33CD5953.021.9590761.0106800.15608591984005700471b2PNA062_unstim_PBMCs_1000cells_S02_S2340.0030742520.022787340.727380
2CD33CD45RB00.080.272660-0.0800000.46811991984005700471b2PNA062_unstim_PBMCs_1000cells_S02_S2340.003074100.000904100.000000
3CD33KLRG110.140.3487350.8600000.19489591984005700471b2PNA062_unstim_PBMCs_1000cells_S02_S2340.003074120.001085120.000000
4CD33VISTA00.200.512471-0.2000000.42074091984005700471b2PNA062_unstim_PBMCs_1000cells_S02_S2340.003074180.001628180.000000
................................................
2679835CD55CD8150.840.9610604.1600000.0000160dd3799c51773670PNA062_unstim_PBMCs_1000cells_S02_S24870.006413320.000421322.321928
2679836CD55IgM104.231.9167263.0103420.0013050dd3799c51773670PNA062_unstim_PBMCs_1000cells_S02_S24870.0064131160.0015281161.241270
2679837CD55VISTA00.410.637150-0.4100000.3409030dd3799c51773670PNA062_unstim_PBMCs_1000cells_S02_S24870.006413110.000145110.000000
2679838CD55HLA-DR-DP-DQ01.671.385750-1.2051240.1140780dd3799c51773670PNA062_unstim_PBMCs_1000cells_S02_S24870.006413460.00060646-0.739848
2679839CD55CD55177.282.2789334.2651540.0000100dd3799c51773670PNA062_unstim_PBMCs_1000cells_S02_S24870.0064134870.0064134871.223524

2679840 rows × 15 columns

Edgelist​

The edge list is a list of detected links between two uniquely identified antibodies. The edge list constitutes the graph which makes up a Proximity Network Assay experiment. Each row contains an edge which is identified by a UMI1 and a UMI2 that are unique identifiers of the antibodies and marker_1 and marker_2 that are the proteins that correspond to the respective antibodies. The edge list is used to infer spatial relationships between antibody molecules, to calculate spatial statistics, and to visualize the cell.

pg_data.edgelist().to_polars()
shape: (99_143_483, 9)
marker_1marker_2umi1umi2read_countuei_countcorrected_read_countcomponentsample
strstru64u64u32u16u32strstr
"CD29""CD43"55563896733551128226901571890329920"8cd2f4585f9102dd""PNA062_unstim_PBMCs_1000cells_…
"B2M""CD44"159299343526092233899519959851077320"8cd2f4585f9102dd""PNA062_unstim_PBMCs_1000cells_…
"B2M""CD5"177270230364296523228591926107523621"8cd2f4585f9102dd""PNA062_unstim_PBMCs_1000cells_…
"CD45""CD45"190503097981205314976911110869850320"8cd2f4585f9102dd""PNA062_unstim_PBMCs_1000cells_…
"CD82""CD82"19278874231014738679240914900276110"8cd2f4585f9102dd""PNA062_unstim_PBMCs_1000cells_…
………………………
"Siglec-9""HLA-DR-DP-DQ"7174650716899867614409997405413253420"f045649c3cc9c83b""PNA062_unstim_PBMCs_1000cells_…
"B2M""CD55"719057712386839634243119471626124110"f045649c3cc9c83b""PNA062_unstim_PBMCs_1000cells_…
"CD371""CD44"7191749663375906469141160293059124110"f045649c3cc9c83b""PNA062_unstim_PBMCs_1000cells_…
"CD44""CD44"7202125816082213662053049232122248420"f045649c3cc9c83b""PNA062_unstim_PBMCs_1000cells_…
"CD44""B2M"7203322320177301311042931600186965330"f045649c3cc9c83b""PNA062_unstim_PBMCs_1000cells_…

Aggregating data​

For cases when we want to process and analyze samples in parallel, it is convenient to merge them into a single object. Here, we read multiple files and aggregate them into a single object. To merge PXL files, we simply pass a list of file paths to the read() function.

from pixelator.pna.pixeldataset.download import DownloadableDatasets

DATA_DIR.mkdir(parents=True, exist_ok=True)

DownloadableDatasets.download_dataset(
"pna062-unstim-pbmcs",
output_path=DATA_DIR / "PNA062_unstim_PBMCs_1000cells_S02_S2.layout.pxl",
)
DownloadableDatasets.download_dataset(
"pna062-pha-pbmcs",
output_path=DATA_DIR / "PNA062_PHA_PBMCs_1000cells_S04_S4.layout.pxl",
)

pg_data_combined = read([
DATA_DIR / "PNA062_unstim_PBMCs_1000cells_S02_S2.layout.pxl",
DATA_DIR / "PNA062_PHA_PBMCs_1000cells_S04_S4.layout.pxl",
])

Now calling any of the functions from the object will include the aggregated data from both samples:

pg_data_combined.adata().obs
n_umi1n_umi2n_edgesreads_in_componentn_antibodiesn_umiisotype_fractionintracellular_fractiontau_typetau...sampleantibodiesaverage_k_corek_core_1k_core_2k_core_3k_core_4svd_var_expl_s1svd_var_expl_s2svd_var_expl_s3
component
f1b52a4758932fc7133551831193749246004133316660.0003160.0normal0.974145...PNA062_unstim_PBMCs_1000cells_S02_S21333.3774715773.03009.05429.08402.00.2844170.1568090.089341
eca4983191f3647f1731123376105495273174139406870.0002210.0normal0.975355...PNA062_unstim_PBMCs_1000cells_S02_S21392.9251857919.05180.09614.017974.00.2535570.1929810.146433
f98240c66b2e49fc118271493478611225033158267610.0017940.0normal0.946812...PNA062_unstim_PBMCs_1000cells_S02_S21583.3730804706.03152.03553.08152.00.2395870.1878860.120709
b5f8b192a0cc2603214502811197857296160158495610.0004040.0normal0.949583...PNA062_unstim_PBMCs_1000cells_S02_S21582.31161612780.012920.019498.04363.00.3493260.2372030.116332
7994e60e303c6dbb1576720905126994357256144366720.0002180.0normal0.952086...PNA062_unstim_PBMCs_1000cells_S02_S21443.8558035937.02389.03245.07657.00.2922640.1730780.140349
..................................................................
fe9e06d663b632d1201952835882112178421156485530.0007000.0normal0.963772...PNA062_PHA_PBMCs_1000cells_S04_S41562.02129613791.019937.014825.00.00.3373050.2112620.172595
b526da61c69412233053739637136362310787153701740.0002140.0normal0.970609...PNA062_PHA_PBMCs_1000cells_S04_S41532.26106516654.018583.034900.037.00.3767070.2827480.156067
8c26b116f942b6133045042223140216307077150726730.0001380.0normal0.942681...PNA062_PHA_PBMCs_1000cells_S04_S41502.25617517579.018898.036196.00.00.4050940.2071560.142091
9567c4b0b99c07ab10783140044490697541141247870.0001210.0normal0.972555...PNA062_PHA_PBMCs_1000cells_S04_S41412.0917827136.08240.09411.00.00.4780560.1632090.079409
1a14cfd48348799d792661077553630227890261581870210.0002830.0normal0.975541...PNA062_PHA_PBMCs_1000cells_S04_S41582.27562142751.049972.094298.00.00.2782790.1916670.181490

2137 rows × 22 columns

Filtering data​

Often we want to constrain our analysis to a subset of the data, e.g. only some samples, components (cells), or markers. This can be done with the .filter() method, which returns a new PixelDataset restricted to the requested selection. Filtering by components is applied consistently across all data modalities (count data, proximity scores, and the edge list), while filtering by samples or markers narrows down the dataset along those dimensions.

As an example, let’s use the n_umi component metadata column to find the components that have at least 10000 molecules detected, and constrain the dataset to only those components. A more thorough quality control procedure, using this and other metrics, is covered in the next tutorial.

component_metadata = pg_data_combined.adata().obs
components_to_keep = component_metadata[component_metadata["n_umi"] >= 10000].index
pg_data_combined_filtered = pg_data_combined.filter(components=components_to_keep)
pg_data_combined_filtered.adata().obs
n_umi1n_umi2n_edgesreads_in_componentn_antibodiesn_umiisotype_fractionintracellular_fractiontau_typetau...sampleantibodiesaverage_k_corek_core_1k_core_2k_core_3k_core_4svd_var_expl_s1svd_var_expl_s2svd_var_expl_s3
component
cdce5da1bac110e71580120226101690273947154360270.0006110.0normal0.957896...PNA062_unstim_PBMCs_1000cells_S02_S21543.2633866091.04262.06716.011983.00.2379260.2042390.156914
f87af461bb0ef75698321395264938188474133237840.0002940.0normal0.978846...PNA062_unstim_PBMCs_1000cells_S02_S21333.1467374297.02795.05604.07297.00.2738890.1812400.077488
c092d3e3cf1d8d3e3500847510157360353925157825180.0002540.0normal0.950072...PNA062_PHA_PBMCs_1000cells_S04_S41572.21977019353.025677.037488.00.00.3556690.2313920.137633
b15677730cdecc8497061122354691123729138209290.0000960.0normal0.960654...PNA062_unstim_PBMCs_1000cells_S02_S21382.9219744141.03461.04375.07794.00.4097620.1761690.072086
7f496640773ed6f3492968302000746621144117590.0005950.0normal0.977429...PNA062_PHA_PBMCs_1000cells_S04_S41442.0266183478.04490.03791.00.00.3527310.1811780.131174
..................................................................
5b29fdc99e17538310998414657256866513007791582565560.0001950.0normal0.967783...PNA062_PHA_PBMCs_1000cells_S04_S41582.60817151468.053280.096118.055690.00.2435940.1688820.120293
f9a6c585f51a36e12649537045125439271851153635400.0002360.0normal0.960348...PNA062_PHA_PBMCs_1000cells_S04_S41532.23980215527.017249.030764.00.00.3469640.1972330.135397
36f1064041e6996a140981885561425137960151329530.0003640.0normal0.978817...PNA062_PHA_PBMCs_1000cells_S04_S41512.1579838868.010011.014074.00.00.3635020.1699810.117631
5a96298f4ccf52f92016722803114857300126144429700.0001630.0normal0.986541...PNA062_unstim_PBMCs_1000cells_S02_S21443.0323717617.04903.08922.021528.00.2846000.1829480.145637
2ecdb279f2f3e3a88834121183895088999154209520.0010980.0normal0.974843...PNA062_PHA_PBMCs_1000cells_S04_S41542.1769765329.06586.09037.00.00.3352900.1761910.152483

2074 rows × 22 columns

The same method can be used to select specific samples or markers, e.g. pg_data_combined.filter(samples="PNA062_PHA_PBMCs_1000cells_S04_S4") to keep only one of the two samples, or pg_data_combined.filter(markers=["CD3e", "CD4"]) to restrict the count and proximity data to a subset of markers.

We have now seen how to load proximity network assay data, inspect its key properties and prepare the data for integrated analysis across experimental samples. In the next tutorial we will apply these skills and demonstrate how to quality control PNA data.