Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation

Francisco Vargas; Ryan Cotterell

Back to publications

Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation

Francisco Vargas, Ryan Cotterell

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020.

Abstract

Bolukbasi et al. (2016) presents one of the first gender bias mitigation techniques for word embeddings. Their method takes pre-trained word embeddings as input and attempts to isolate a linear subspace that captures most of the gender bias in the embeddings. As judged by an analogical evaluation task, their method virtually eliminates gender bias in the embeddings. However, an implicit and untested assumption of their method is that the bias sub-space is actually linear. In this work, we generalize their method to a kernelized, non-linear version. We take inspiration from kernel principal component analysis and derive a non-linear bias isolation technique. We discuss and overcome some of the practical drawbacks of our method for non-linear gender bias mitigation in word embeddings and analyze empirically whether the bias subspace is actually linear. Our analysis shows that gender bias is in fact well captured by a linear subspace, justifying the assumption of Bolukbasi et al. (2016).

Links

Cite this Paper

BibTeX


@InProceedings{publications/exploring-the-linear-subspace-hypothesis-in-gender-bias-mitigation,
  title = 	 {Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation},
  author = 	 {Vargas, Francisco and Cotterell, Ryan},
  booktitle = 	 {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing},
  year = 	 {2020},
  url = 	 {/publications/exploring-the-linear-subspace-hypothesis-in-gender-bias-mitigation.html},
  abstract = 	 {Bolukbasi et al. (2016) presents one of the first gender bias mitigation techniques for word embeddings. Their method takes pre-trained word embeddings as input and attempts to isolate a linear subspace that captures most of the gender bias in the embeddings. As judged by an analogical evaluation task, their method virtually eliminates gender bias in the embeddings. However, an implicit and untested assumption of their method is that the bias sub-space is actually linear. In this work, we generalize their method to a kernelized, non-linear version. We take inspiration from kernel principal component analysis and derive a non-linear bias isolation technique. We discuss and overcome some of the practical drawbacks of our method for non-linear gender bias mitigation in word embeddings and analyze empirically whether the bias subspace is actually linear. Our analysis shows that gender bias is in fact well captured by a linear subspace, justifying the assumption of Bolukbasi et al. (2016).}
}

Endnote

%0 Conference Paper
%T Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation
%A Francisco Vargas
%A Ryan Cotterell
%B Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing
%D 2020	
%F publications/exploring-the-linear-subspace-hypothesis-in-gender-bias-mitigation
%U /publications/exploring-the-linear-subspace-hypothesis-in-gender-bias-mitigation.html
%X Bolukbasi et al. (2016) presents one of the first gender bias mitigation techniques for word embeddings. Their method takes pre-trained word embeddings as input and attempts to isolate a linear subspace that captures most of the gender bias in the embeddings. As judged by an analogical evaluation task, their method virtually eliminates gender bias in the embeddings. However, an implicit and untested assumption of their method is that the bias sub-space is actually linear. In this work, we generalize their method to a kernelized, non-linear version. We take inspiration from kernel principal component analysis and derive a non-linear bias isolation technique. We discuss and overcome some of the practical drawbacks of our method for non-linear gender bias mitigation in word embeddings and analyze empirically whether the bias subspace is actually linear. Our analysis shows that gender bias is in fact well captured by a linear subspace, justifying the assumption of Bolukbasi et al. (2016).

RIS


TY  - CPAPER
TI  - Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation
AU  - Francisco Vargas
AU  - Ryan Cotterell
BT  - Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing
DA  - 2020/11/17	
ID  - publications/exploring-the-linear-subspace-hypothesis-in-gender-bias-mitigation
UR  - /publications/exploring-the-linear-subspace-hypothesis-in-gender-bias-mitigation.html
AB  - Bolukbasi et al. (2016) presents one of the first gender bias mitigation techniques for word embeddings. Their method takes pre-trained word embeddings as input and attempts to isolate a linear subspace that captures most of the gender bias in the embeddings. As judged by an analogical evaluation task, their method virtually eliminates gender bias in the embeddings. However, an implicit and untested assumption of their method is that the bias sub-space is actually linear. In this work, we generalize their method to a kernelized, non-linear version. We take inspiration from kernel principal component analysis and derive a non-linear bias isolation technique. We discuss and overcome some of the practical drawbacks of our method for non-linear gender bias mitigation in word embeddings and analyze empirically whether the bias subspace is actually linear. Our analysis shows that gender bias is in fact well captured by a linear subspace, justifying the assumption of Bolukbasi et al. (2016).
ER  -

APA


Vargas, F. & Cotterell, R.. (2020). Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing Available from /publications/exploring-the-linear-subspace-hypothesis-in-gender-bias-mitigation.html.