120 lines
4.7 KiB
Markdown
120 lines
4.7 KiB
Markdown
# Deep Learning in Mass Spectrometry Imaging: A Hybrid Julia-Python Workflow
|
|
|
|
This repository contains the computational pipeline for the automated anatomical mapping of *Datura innoxia* (toloache) using Mass Spectrometry Imaging (MSI). The workflow addresses the "two-language problem" by leveraging **Julia** for high-performance data preprocessing and **Python** for self-supervised representation learning.
|
|
|
|
## Key Features
|
|
- **High-Performance Preprocessing**: Rapid `.imzML` ingestion and TrIQ normalization using Julia.
|
|
- **Self-Supervised Learning**: Contrastive learning (SimCLR) with an EfficientNet-B0 backbone implemented in PyTorch.
|
|
- **Reproducibility**: Integrated Docker environment and fixed random seeds for consistent results.
|
|
- **Performance Benchmarking**: Real-time monitoring of execution time and RAM usage across the hybrid pipeline.
|
|
- **CUDA/GPU Support**: Optimized for NVIDIA hardware for accelerated training and inference.
|
|
- **Hybrid Bridge**: Automated data transfer between Julia and Python ecosystems via standardized intermediate files.
|
|
|
|
## Project Structure
|
|
- `scripts_julia/`: Julia scripts for mass slice extraction and TrIQ normalization.
|
|
- `scripts_python/`: Python scripts for SimCLR training, feature extraction, and clustering.
|
|
- `models/`: Contains the pre-trained weights (`msi_encoder_trained.pth`).
|
|
- `environment/`: Dependency files (`requirements.txt`, `Project.toml`).
|
|
- `figures/`: Generated plots and cluster galleries.
|
|
- `data/`: (User-provided) Location for `.imzML` and mask files.
|
|
|
|
## Reproducibility & Docker (Recommended)
|
|
To ensure identical environments for both Julia and Python, we provide a unified Docker setup. This setup includes the **JuliaMSI** framework.
|
|
|
|
### Building the Container
|
|
Before building, ensure the `JuliaMSI` repository is present in the root directory (it was cloned from `https://git.nube-gran.de/Julian/JuliaMSI`).
|
|
```bash
|
|
docker build -t msi-hybrid-workflow .
|
|
```
|
|
|
|
### Running the Container (CPU Mode)
|
|
```bash
|
|
docker run -it --rm -v $(pwd):/app msi-hybrid-workflow
|
|
```
|
|
|
|
### Running the Container (GPU/CUDA Mode)
|
|
```bash
|
|
docker run -it --rm --shm-size=2gb --gpus all -v $(pwd):/app msi-hybrid-workflow
|
|
```
|
|
|
|
## Usage Workflow
|
|
|
|
### 1. Preprocessing (Julia)
|
|
Leverage Julia's performance and the **JuliaMSI** framework.
|
|
|
|
**Run this clean runtime instantiation command one time to sync everything down to your local directory:**
|
|
```bash
|
|
docker run -it --rm -v $(pwd):/app msi-hybrid-workflow \
|
|
julia --project=/app/environment -e 'using Pkg; Pkg.resolve(); Pkg.instantiate()'
|
|
```
|
|
|
|
**Run the MSI Bridge (Normalization & PNG Export):**
|
|
```bash
|
|
docker run -it --rm --gpus all -v $(pwd):/app msi-hybrid-workflow \
|
|
julia --threads auto --project=/app/environment /app/scripts_julia/julia_msi_bridge.jl
|
|
```
|
|
|
|
**Optional: Plot Cluster Spectra:**
|
|
```bash
|
|
docker run -it --rm --gpus all -v $(pwd):/app msi-hybrid-workflow \
|
|
julia --threads auto --project=/app/environment scripts_julia/graph_cluster_spectra.jl
|
|
```
|
|
|
|
### 2. Representation Learning (Python)
|
|
Train the SimCLR encoder to extract morphological features using CUDA.
|
|
|
|
**Run CNN Training (SimCLR):**
|
|
```bash
|
|
docker run -it --rm \
|
|
--shm-size=2gb \
|
|
--gpus all \
|
|
-v $(pwd):/app \
|
|
-v msi_torch_cache:/root/.cache/torch \
|
|
msi-hybrid-workflow \
|
|
python3 scripts_python/CNN_proof_1.py
|
|
```
|
|
*Note: The script automatically detects CUDA and logs per-epoch RAM usage.*
|
|
|
|
### 3. Clustering & Validation
|
|
Determine optimal clusters and perform quantitative validation.
|
|
|
|
**Generate Clusters:**
|
|
```bash
|
|
docker run -it --rm --shm-size=2gb --gpus all -v $(pwd):/app msi-hybrid-workflow \
|
|
python3 scripts_python/cluster_msi_direct.py
|
|
```
|
|
|
|
**Evaluate Clustering:**
|
|
```bash
|
|
docker run -it --rm --gpus all -v $(pwd):/app msi-hybrid-workflow \
|
|
python3 scripts_python/evaluate_clustering.py
|
|
```
|
|
|
|
**Quantitative Validation:**
|
|
```bash
|
|
docker run -it --rm --gpus all -v $(pwd):/app msi-hybrid-workflow \
|
|
python3 scripts_python/quantitative_validation.py
|
|
```
|
|
|
|
### 4. Visualization
|
|
Generate the final galleries and t-SNE maps.
|
|
|
|
**Generate t-SNE Maps:**
|
|
```bash
|
|
docker run -it --rm --shm-size=2gb --gpus all -v $(pwd):/app msi-hybrid-workflow \
|
|
python3 scripts_python/graficar_tsne_msi.py
|
|
```
|
|
|
|
**Create Cluster Galleries:**
|
|
```bash
|
|
docker run -it --rm --shm-size=2gb --gpus all -v $(pwd):/app msi-hybrid-workflow \
|
|
python3 scripts_python/visualize_clusters.py
|
|
```
|
|
|
|
## Citation & Acknowledgments
|
|
This implementation is based on the methodology proposed by:
|
|
> Hu, H., Bindu, J. P., & Laskin, J. (2022). Self-supervised clustering of mass spectrometry imaging data using contrastive learning. *Chemical Science*, 13(1), 90-98. DOI: [10.1039/d1sc04077d](https://doi.org/10.1039/d1sc04077d)
|
|
|
|
## License
|
|
This project is licensed under the **MIT License**.
|