선동이의 삶

목차 iBOT: Image BERT Pre-Training with Online Tokenizer ICLR 2022 https://arxiv.org/abs/2111.07832 iBOT: Image BERT Pre-Training with Online Tokenizer The success of language Transformers is primarily attributed to the pretext task of masked language modeling (MLM), where texts are first tokenized into semantically meaningful pieces. In this work, we study masked image modeling (MIM) and indicate..

목차 Emerging Properties in Self-Supervised Vision Transformers ICCV 2021 https://arxiv.org/abs/2104.14294 Emerging Properties in Self-Supervised Vision Transformers In this paper, we question if self-supervised learning provides new properties to Vision Transformer (ViT) that stand out compared to convolutional networks (convnets). Beyond the fact that adapting self-supervised methods to this arc..