“Disentangling” in ICLR2021
ICLR2021 openreivew 下搜索 “dientangling” 关键字的文章共计 92 篇，我先大致阅读标题、摘要、结论，对文章进行分类、质量筛选，选出几篇比较有特点的文章，且值得深入阅读的文章。
更新到第10条。
Paper-List
- Disentangling Action Sequences: Discovering Correlated Samples. (checked)
本文提出了一个 Action Sequences 的概念：样本集中的关于某种 transformation 的因子连续变化的子集。 但是后续论文的写作上非常含糊，很难读懂，并且在 FVAE 的模型上，也难看出作者应用的思想。 但是作者给了代码，推荐结合代码阅读。
- Disentangled Representations from Non-Disentangled Models. (checked)
本文希望从 pre-trained GAN 中提取 diensentangled 表示。 两阶段法：首先使用 Closed-Form 等无监督学习方法对预训练的GAN获取隐空间的正交的移动方向，然后将隐变量在这些正交方向上的投影看成 disentangled 的表示，以此训练一个 Encoder。 除此之外，作者提出了一种新的方向寻找方法: 使用更高层的输出的 Jacobian 矩阵的主成分。
我曾经也设想过这种从 GAN 中提取 dientanlged 表示的思路，但是本文的创新性还是比较低，比较水。
- Multi-View Disentangled Representation.
tags: 多视角数据disentangled表示学习;
本文为无监督的多视角表示学习下了数学定义。 并基于互信息量提出了无监督的方法。
- Disentangled Recurrent Wasserstein Autoencoder. [paper]
tags: 序列数据disentangled表示学习;
本文将序列数据分为时间不变(static)和时间改变(dynamic)的表示。
- ADIS-GAN: Affine Disentangled GAN. [paper]
tags: GAN-based; affine transfomation disentangled;
本文提出了通过自监督训练可以 disentangle 放射变换的 GAN 结构(ADIS-GAN)。 从结构上说，作者也是加入了一个 regressor 预测自监督中单个改变的放射变化的尺度（文中是变换的矩阵）。
我认为总的来说，这种关于仿射变换的自监督方式的工作确实比较多了，稍显创新性不足。 简单来说就是略水。 从效果上来说，InfoGAN-CR 比本文的效果略胜一筹，而且不局限于仿射变换因子的 disentangle，是本文的一大弱点。
- On the Transfer of Disentangled Representations in Realistic Settings. [paper]
tags: large scale; real-world; dataset; VAE-based;
本文旨在将 DRL 扩展到大规模、真实世界。首先作者基于 3D 的机器人场景构建了大规模（超过1M张图像）的真实数据集（提供1K的少量的标注）；其次在深度扩展原始的VAE模型，提出了一种 ResNet 架构的VAE用于该种任务。
这熟悉的图表样式，一看不就是 Locatello 组的 paper 吗？双盲也不过如此。。 该组深耕 DRL 比较久，成果也比较多，从 paper 上看，他们组应该准备向大规模、真实场景的弱监督 DRL 方向研究。 该组的文章的理论、实验上都比较严谨，值得一看。 值得一提的是，我认为 VAE 在 DRL 的一大限制就是 VAE 的重建能力，现在的 VAE 模型普遍偏小，并且生成图像质量并不高，远低于GAN，这可能一方面是模型容量的影响，另一方面是 KL divergence 优化的 trade-off 问题，但是如果学习不到真实的图像的重建，DR 其实也学习地并不好。
- Disentangled Generative Causal Representation Learning. [paper]
tags: causality;
本文将 underlying 因子的因果关系带入考虑学习 DR。 作者将 SCM (结构因果方程) 作为隐变量的先验训练双向生成模型。
没有看文章的方法，但是本文似乎是通过改变先验的方式，以往改变先验形式的工作也不少，这种工作的关键在于： (1) 先验形式(分布族)的选择, 参数; (2) 如何采样和进行重参数化。
- Unsupervised Disentanglement Learning by intervention. [paper]
tags: correlated; unsupervised; self-supervised;
本文考虑了真实因子相关的 DRL 的无监督方法。 其实作者还是采用了自监督的方法。从模型上看，作者也采用了 latent code 交换进行重建的思路，不同的是作者再交换一次，换回原用的 code，引入第二个重建 loss。 除此之外作者还加入了 adv loss 和 diff loss。（未细看）
- On Disentangled Representations Learned From Correlated Data. [[paper]]
tags: large scale; correlated;
本文同样考虑真实因子相关的 DRL 问题，并旨在将模型推向大规模数据上，不过本文主要通过实验检验了 independence-induced 的无监督 以及半监督、弱监督方法、以及现有的 meterics 在真实因子相关的数据集上的表现。 可以想到他们的表现都不太好，作者继而提出了一种弱监督的改进方法。最后作者认为本文表明，对 DRL，寻找学习独立的生成机制 (independent mechanism) 比起寻找单纯的变化因子更为重要。
嗯，这也是 Locatello 组的。 得看。
- DynamicVAE: Decoupling Reconstruction Error and Disentangled Representation Learning. [paper]
本文是 ControlVAE 的改进。 引入增量PI控制器控制 ，从而更好地平衡 -VAE 的重建 loss 和 KL loss 的 trade-off。
“Challenge” 一文验证了几个经典的无监督模型实质上的表现差别不大，所以本文作者注重 -VAE 的调节也不失重要性。 本文的方法也许对改善 VAE 的生成能力也有帮助，可以一看。
-
Disentangling Representations of Text by Masking Transformers.
-
GL-Disen: Global-Local disentanglement for unsupervised learning of graph-level representations. [paper]
Addressing the Topological Defects of Disentanglement.
Learning to Disentangle Textual Representations and Attributes via Mutual Information
GraphSAD: Learning Graph Representations with Structure-Attribute Disentanglement
RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior
Generating Furry Cars: Disentangling Object Shape and Appearance across Multiple Domains
Generative Auto-Encoder: Non-adversarial Controllable Synthesis with Disentangled Exploration
Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modelling
Sufficient and Disentangled Representation Learning
The role of Disentanglement in Generalisation
Clearing the Path for Truly Semantic Representation Learning
{Learning disentangled representations with the Wasserstein Autoencoder
Quantifying and Learning Disentangled Representations with Limited Supervision. [paper]
tags: weak supervised;
本文根据 Higgins2018 的 Linear Symmetry-based Disentanglement(LBSD) 表示的评价方法，然后
将原有的方法扩展为弱监督类型。
Evaluating the Disentanglement of Deep Generative Models through Manifold Topology. [paper]
tags: evaluation;
Learning Disentangled Representations for Image Translation. [paper]
tags: application; image translation;
Disentanglement, Visualization and Analysis of Complex Features in DNNs
Information Theoretic Regularization for Learning Global Features by Sequential VAE
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
What’s new? Summarizing Contributions in Scientific Literature
Disentangled cyclic reconstruction for domain adaptation
Improving the Unsupervised Disentangled Representation Learning with VAE Ensemble
Untangle: Critiquing Disentangled Recommendations
PDE-Driven Spatiotemporal Disentanglement
Improving VAEs’ Robustness to Adversarial Attack
Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
Efficiently Disentangle Causal Representations
Identifying Informative Latent Variables Learned by GIN via Mutual Information
On the Role of Pre-training for Meta Few-Shot Learning
Rethinking Content and Style: Exploring Bias for Unsupervised Disentanglement
Disentangling 3D Prototypical Networks for Few-Shot Concept Learning
Identifying Coarse-grained Independent Causal Mechanisms with Self-supervision
Learning from Demonstration with Weakly Supervised Disentanglement
CIGMO: Learning categorical invariant deep generative models from grouped data
Difference-in-Differences: Bridging Normalization and Disentanglement in PG-GAN
DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION
Disentangling Adversarial Robustness in Directions of the Data Manifold
Toward Understanding Supervised Representation Learning with RKHS and GAN
Shape or Texture: Disentangling Discriminative Features in CNNs
Batch Normalization Increases Adversarial Vulnerab…gling Usefulness and Robustness of Model Features
Compositional Video Synthesis with Action Graphs
ProGAE: A Geometric Autoencoder-based Generative Model for Disentangling Protein Dynamics
Self-supervised Disentangled Representation Learning
Towards Robust Textual Representations with Disentangled Contrastive Learning
Disentangling style and content for low resource v…tion: a case study on keystroke inference attacks
Category Disentangled Context: Turning Category-irrelevant Features Into Treasures
Improving Zero-Shot Voice Style Transfer via Disentangled Representation Learning
Noise or Signal: The Role of Image Backgrounds in Object Recognition
Intrinsically Guided Exploration in Meta Reinforcement Learning
Learned ISTA with Error-based Thresholding for Adaptive Sparse Coding
Zero-shot Fairness with Invisible Demographics
PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences
Model-Free Counterfactual Credit Assignment
Failure Modes of Variational Autoencoders and Their Effects on Downstream Tasks
Understanding, Analyzing, and Optimizing the Complexity of Deep Models
Unsupervised Discovery of 3D Physical Objects
Representation learning for improved interpretabil…ssification accuracy of clinical factors from EEG
Model-based Navigation in Environments with Novel Layouts Using Abstract 2-D Maps
Identifying the Sources of Uncertainty in Object Classification
CURI: A Benchmark for Productive Concept Learning Under Uncertainty
AutoBayes: Automated Bayesian Graph Exploration for Nuisance-Robust Inference
Property Controllable Variational Autoencoder via Invertible Mutual Dependence
Latent Causal Invariant Model
Domain-Robust Visual Imitation Learning with Mutual Information Constraints
Self-supervised Visual Reinforcement Learning with Object-centric Representations
Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning
How to Train Your Super-Net: An Analysis of Training Heuristics in Weight-Sharing NAS
Towards Noise-resistant Object Detection with Noisy Annotations
Unsupervised Learning of Global Factors in Deep Generative Models
Language-Mediated, Object-Centric Representation Learning
Time-varying Graph Representation Learning via Higher-Order Skip-Gram with Negative Sampling
Hierarchical Meta Reinforcement Learning for Multi-Task Environments
Cross-Node Federated Graph Neural Network for Spatio-Temporal Data Modeling
Counterfactual Generative Networks
Targeted VAE: Structured Inference and Targeted Learning for Causal Parameter Estimation
Multi-Class Uncertainty Calibration via Mutual Information Maximization-based Binning
Zero-shot Synthesis with Group-Supervised Learning
Max-Affine Spline Insights Into Deep Generative Networks
Human-interpretable model explainability on high-dimensional data
Spatial Dependency Networks: Neural Layers for Improved Generative Image Modeling
On the role of planning in model-based deep reinforcement learning
Asynchronous Modeling: A Dual-phase Perspective for Long-Tailed Recognition
A Good Image Generator Is What You Need for High-Resolution Video Synthesis.
tags: application; video generation;
Information Lattice Learning.
CaLFADS: latent factor analysis of dynamical systems in calcium imaging data
Beyond Trivial Counterfactual Generations with Diverse Valuable Explanations