Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis

Research output: Working paper › Preprint › Research

Standard

Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis. / Schön, Julian; Selvan, Raghavendra; Nygård, Lotte; Vogelius, Ivan Richter; Petersen, Jens.

arxiv.org, 2023.

Research output: Working paper › Preprint › Research

Harvard

Schön, J, Selvan, R, Nygård, L, Vogelius, IR & Petersen, J 2023 'Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis' arxiv.org. https://doi.org/10.48550/arXiv.2301.05465

APA

Schön, J., Selvan, R., Nygård, L., Vogelius, I. R., & Petersen, J. (2023). Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis. arxiv.org. https://doi.org/10.48550/arXiv.2301.05465

Vancouver

Schön J, Selvan R, Nygård L, Vogelius IR, Petersen J. Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis. arxiv.org. 2023 Jan 13. https://doi.org/10.48550/arXiv.2301.05465

Author

Schön, Julian ; Selvan, Raghavendra ; Nygård, Lotte ; Vogelius, Ivan Richter ; Petersen, Jens. / Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis. arxiv.org, 2023.

Bibtex

@techreport{599dfaa5faf148329401953884fbf88e,
title = "Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis",
abstract = "Medical imaging plays a vital role in modern diagnostics and treatment. The temporal nature of disease or treatment progression often results in longitudinal data. Due to the cost and potential harm, acquiring large medical datasets necessary for deep learning can be difficult. Medical image synthesis could help mitigate this problem. However, until now, the availability of GANs capable of synthesizing longitudinal volumetric data has been limited. To address this, we use the recent advances in latent space-based image editing to propose a novel joint learning scheme to explicitly embed temporal dependencies in the latent space of GANs. This, in contrast to previous methods, allows us to synthesize continuous, smooth, and high-quality longitudinal volumetric data with limited supervision. We show the effectiveness of our approach on three datasets containing different longitudinal dependencies. Namely, modeling a simple image transformation, breathing motion, and tumor regression, all while showing minimal disentanglement. The implementation is made available online at https://github.com/julschoen/Temp-GAN.",
keywords = "cs.CV, cs.LG",
author = "Julian Sch{\"o}n and Raghavendra Selvan and Lotte Nyg{\aa}rd and Vogelius, {Ivan Richter} and Jens Petersen",
year = "2023",
month = jan,
day = "13",
doi = "10.48550/arXiv.2301.05465",
language = "English",
publisher = "arxiv.org",
type = "WorkingPaper",
institution = "arxiv.org",

}

RIS

TY - UNPB

T1 - Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis

AU - Schön, Julian

AU - Selvan, Raghavendra

AU - Nygård, Lotte

AU - Vogelius, Ivan Richter

AU - Petersen, Jens

PY - 2023/1/13

Y1 - 2023/1/13

N2 - Medical imaging plays a vital role in modern diagnostics and treatment. The temporal nature of disease or treatment progression often results in longitudinal data. Due to the cost and potential harm, acquiring large medical datasets necessary for deep learning can be difficult. Medical image synthesis could help mitigate this problem. However, until now, the availability of GANs capable of synthesizing longitudinal volumetric data has been limited. To address this, we use the recent advances in latent space-based image editing to propose a novel joint learning scheme to explicitly embed temporal dependencies in the latent space of GANs. This, in contrast to previous methods, allows us to synthesize continuous, smooth, and high-quality longitudinal volumetric data with limited supervision. We show the effectiveness of our approach on three datasets containing different longitudinal dependencies. Namely, modeling a simple image transformation, breathing motion, and tumor regression, all while showing minimal disentanglement. The implementation is made available online at https://github.com/julschoen/Temp-GAN.

AB - Medical imaging plays a vital role in modern diagnostics and treatment. The temporal nature of disease or treatment progression often results in longitudinal data. Due to the cost and potential harm, acquiring large medical datasets necessary for deep learning can be difficult. Medical image synthesis could help mitigate this problem. However, until now, the availability of GANs capable of synthesizing longitudinal volumetric data has been limited. To address this, we use the recent advances in latent space-based image editing to propose a novel joint learning scheme to explicitly embed temporal dependencies in the latent space of GANs. This, in contrast to previous methods, allows us to synthesize continuous, smooth, and high-quality longitudinal volumetric data with limited supervision. We show the effectiveness of our approach on three datasets containing different longitudinal dependencies. Namely, modeling a simple image transformation, breathing motion, and tumor regression, all while showing minimal disentanglement. The implementation is made available online at https://github.com/julschoen/Temp-GAN.

KW - cs.CV

KW - cs.LG

U2 - 10.48550/arXiv.2301.05465

DO - 10.48550/arXiv.2301.05465

M3 - Preprint

BT - Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis

PB - arxiv.org

ER -

ID: 333626101