多模态医学影像与人工智能.docx

G库建弓）多毓、多中心数据库：20000余例乳腺癌、头颈肿瘤、抑郁症北京协和医院、北京大学肿瘤医院、复旦大学肿瘤医院等近20家大数据人工智能科研平台微据与人工智惚实险宦I程大鹏的屏幕共享 CESM（对比增强能谱X线摄影） 2017年至今，已完成IoOoO余例微据易人工智健实龄室程大鹏的屏幕共享研究思路第一部分肿瘤诊断第二部分I预测淋巴结转移第三部分预测病理分型Q第四部分预测新辅助化疗疗效第五部分预后预测I第六部分预测术后复发风险预测基因表达息程大鹏的屏幕共享02Scientific Research影像组学:A.图像获取公OlBL病灶分割 ProstateStruct resNormal Appearing) TiSSC.特征提取D.特征筛选E.模型构建影像组学：高通量地提取海量影像信息，实现肿瘤分割、特征提取与模型建 立，凭借对海量影像数据信息进行更深层次的挖掘、分析和预测，辅助医师 做出准确的诊断息程大鹏的破据5国工留唯险屏幕共享 02 Scientific Research第一部分：深度学习：乳腺肿瘤分类 A流程图Mao N,et al.British Journal of Cancer,2022 （中科院 1 区，IF=9）3微据与人工智惚实险室程大鹏的屏幕共享全自动管道系统对乳腺肿瘤进行自动分割和分类Adaptive ConvEntry flowRefineNetMuti-resolution fusion Chained Residual pooling Output ConvMiddle flowXceptionExit flowPPMGlobal averagepoolingSeparable BlockPyramid poolingmodule(PPM)Zheng T, et al. eClinicalMedicine, 2023 (柳叶刀子刊，中科院1 区，IF=I7 ) t程大鹏的屏幕共享第二部分：基于DCE-MRI深度学习的乳腺癌自动分割、诊断和淋巴结转移预测B第一阶段段对病灶实现自动分割第二阶段对分割的病灶进行良恶性分类对分类为恶性的患者进行淋巴结状态预测OQO Ooooo OQOOo教据鸟曷工留穗龄生痣|同程大鹏的屏幕共享第三部分：瘤内和瘤周影像组学对乳腺癌新辅助化疗(NAC)早期预测(1)瘤内(2)瘤周5mm(3)瘤周IOmm瘤内+瘤周5mm(5)瘤内+瘤周IommMao Met al.Eur Radiol,2022熬据与人工智怩实验宦 $ 的鹏的屏幕共享第三部分：MRl-病理WSl多模态融合预测乳腺癌新辅助化疗(NAC)疗效a.MRI标签的构建b.WSl 标签时科建机器学习d.集成模型的构建达到PCR'未达-到PCR注意力主干网络注意力分支网络临床信息：分子分型、年龄、分期等一 C. 8 程大鹏的屏幕共享脑成像+人工智能功能成像活动模式网络模式程大鹏的屏幕共享多模态 S)网络融合救辗与人工留犍原龄蜜脑功能与结构网络变化.模式识别Ii 与预测疾病影像标记脑功能成像+机器学习me! *'*A7ly1V*ij<1iVju/''c Fature estraction and selectionYwwsn小winit基于时间和空间变异性特征，诊断抑郁症Gai Q,et al.J Magn Reson Imaging5 2022同程大鹏的屏幕共享研允热点影像+病理Image acqulstonLancet Digit Health 2022ArticlesDevelopment and validation of a radiopathomics model to 、® predict pathological complete response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: a multicentre observational studyLiifengn.Zhenw Liu19ChaofengL1,Zhenhuili1,XiaoyingLou.Lizhi Shaot9Yundlong Wang Yan Huang Haiyang Chen9Xiadin Pong ShuaiLio.Fang He9Jian Zheng.Xiaoxhun Meng PeiyiXie9GuanyuYangYiDing MingbiaoWeijingping Yun9Mien-Chie HungWethua Zhou.Danid RWaht(PingLant,jie Tiant9Xiangbo WantoaIo-080604184tanememo Ohmphr-Kapan-Meiet analps-MefctedgCR-edicted nen pRTime (monthn)一), 网 SF国；口20 4060 80 100Modal Constrction Signature integationMB sqnutumthtersthednCRTStantand CRT结合磁共振图像和病理全切片成像(WSI),采用影像病理组 学分析，区分直肠癌新辅助化疗的不同反应，结果显示多模 态AI模型高于单一模态模型(单纯影像或病理图像)Figre I Wonkflow Ofthestudy据智犍U tKz程大鹏的屏幕共享录制中FATI(MOMA)FArnIPC-OHlP)UNC00290 (MOMA)LINC00290IPC-CHP)PTEN(MOMA) PtEN(PC-CHIP)nature communicationsArticleFHHT(PC-CHiP)PPP2R2A(MOMA) PPP2R2A(PC-CHPMACR0D2(M0MA)MACROD2 (PC-CHiP)Cshdi(MOMA)Histopatholog. aberrations an cancer patientAReceived:4 August 2022Accepted :3 March 2023Published online: 13 April 2023nature communicationsArtiChttps doi. org10.1038s41467-023-43172-8Deep learning of cell spatial organizations identifies clinically relevant insights in tissue imagesAUROCREAD DelCheck for updatesModel prediction»Mun-abelCaSSineaton MedelResults of 7 conceptsCancer assocated stroma fSTR)=N Lymghocytes LVM): Mucun (MUC):%Colorectal adenocarcinoma epithelium (TUM):Tiasue debts (DEB):Smooth musde (MUS): Adipore tihsue(ADH):Traditional model such as CNNTumor S Lymphocyte StromalCellMacrophage Karyorrhexis Rod Blood Cell研究热点U环境二）nature communicationsArticlehtpsdoi org10. 1038s41467-023-43172-8Deep learning of cell spatial organizations identifies clinically relevant insights in tissue imagesReceived:24 June 2023Accepted:2 November 2023Published online: 11 December 2023Shidan Wang 1 ,Ruichen Rong',Qin Zhou',Donghan M.Yang',Xinyi Zhang !,Xia。Wei Zhan',Justin Bishop2jZhikai Chi O2,Clare LWiIheIm2,Siyuan Zhang ?,Curtis R.Pickering*,Mark G.KrisjJohn Minna5.87,Yang Xieta3ScGuanghua Xiaoola3Check for updatesCeographCell spatial graphCSlGC GContributionInterpretCSlGCPredictLabeln=number of classes/人撤据与人工智健实龄室鎏 程大鹏的屏幕共享Radiologists pay more attention.The arterial phaselTimeirQRadiologists'Attention WeightSelf-learning TemporalAttentiOll ModUleC×T×HxWExtract Key Frame1OOOConcatenateBrightness Change CurveI BackboneC×T×H×W一Mmq扁"一 HDomain knowledge Domain Knowledge Guided /Cwedeo ”Classification Score(b)Brightness Change