Posts by Tag

• Machine Learning 41
• Paper 12
• Reading 8
• NLP 4
• Transformer 3
• Parameter Efficiency 2
• xgboost 1
• computer_vision 1
• parallelization 1
• audio 1
• self_supervised_learning 1
• parameter_efficiency 1
• adapters 1
• Transformers 1
• finetuning 1
• SentenceEmbeddings 1
• Prompting 1
• Few Shot Learning 1
• Soft Prompting 1
• Paper Summaries 1

Machine Learning

1 Year of a Challenging Big-Bench Task

16 minute read

In 2021 I contributed to the Big-Bench suite of NLP tasks, aiming to probe the abilities of large language models. Inspired by sports, I developed a task aim...

Contrastive Training Improves Zero-Shot Classification of

less than 1 minute read

Introduction

On Transferability of Prompt Tuning for Natural Language Processing

1 minute read

Introduction

Relation Extraction as Open-book Examination:

1 minute read

Introduction

Document-level Relation Extraction as Semantic Segmentation

1 minute read

Introduction

Ontology-enhanced Prompt-tuning for Few-shot Learning

1 minute read

Introduction

Trial2Vec

1 minute read

Trial2Vec

DocFormer: End-to-End Transformer for Document Understanding

2 minute read

Introduction

TeKo: Text-Rich Graph Neural Networks

1 minute read

Introduction

Prompt for Extraction? PAIE: Prompting Argument Interaction for

1 minute read

Introduction

Domain-matched Pre-training Tasks for Dense Retrieval

1 minute read

Introduction

Universal Sentence Representation Learning with Conditional Masked Language Model

1 minute read

Introduction

TABi: Type-Aware Bi-Encoders for Open-Domain Entity Retrieval

1 minute read

Introduction

CDLM: Cross-Document Language Model

1 minute read

Introduction

SPANBERT

less than 1 minute read

What is the name of the SpanBERT paper? SpanBERT: Improving Pre-training by Representing and Predicting Spans Mandar Joshi, ...

UNDERSTANDING DIMENSIONAL COLLAPSE IN CONTRASTIVE SELF-SUPERVISED LEARNING

1 minute read

Introduction

Embedding Hallucination for Few-Shot Language Fine-tuning

less than 1 minute read

Introduction

Contrastive Learning for Prompt-Based Few-Shot Language Learners

1 minute read

Introduction

Condenser

1 minute read

Introduction

Adaptable Adapters

less than 1 minute read

Introduction

Improving Multi-task Generalization Ability for Neural Text

1 minute read

What is the name of the MatchPrompt paper? Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning ...

Introducing Neural Bag of Whole-Words with ColBERTer

1 minute read

Introduction

DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings

2 minute read

Introduction

Exploring Dual Encoder Architectures for Question Answering

less than 1 minute read

Introduction

Dense Passage Retrieval for Open-Domain Question Answering

1 minute read

Introduction

ARE NEURAL NETS MODULAR? INSPECTING FUNCTIONAL MODULARITY THROUGH DIFFERENTIABLE

1 minute read

machinelearning::papers::AreNeuralNetworksModular

Evaluating Distributional Distortion in Neural Language Modeling

1 minute read

Introduction

A Structural Probe for Finding Syntax in Word Representations

1 minute read

Introduction

Hyperlink-induced Pre-training

1 minute read

Introduction

LinkBERT: Pretraining Language Models with Document Links

1 minute read

Introduction

Cluster & Tune: Boost Cold Start Performance in Text Classification

1 minute read

Introduction

Token Dropping for Efficient BERT Pretraining

1 minute read

Introduction

Phrase-BERT: Improved Phrase Embeddings from BERT with an to Corpus Exploration

2 minute read

Introduction

Memorizing Transformers

2 minute read

Introduction

Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models

3 minute read

Introduction

Deep Continuous Prompt for Contrastive Learning of Sentence Embeddings

2 minute read

Introduction

UCTopic: Unsupervised Contrastive Learning for Phrase Representations

2 minute read

Introduction

Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models

2 minute read

Introduction

Capturing Failures of Large Language Models via Human Cognitive Biases

3 minute read

Intro

T5: Exploring the limits of Transfer Learning with a Unified Text to Text Transformer

4 minute read

Introduction

TransformerXL

2 minute read

Introduction Transformer models typically have a fixed context window that is hard to scale due to the $O(n^2)$ cost of the attention mechanism. Extending th...

Back to Top ↑

Paper

Decoder Inference Optimization

5 minute read

Introduction

The Role of Complex NLP in Transformers for Text Ranking

less than 1 minute read

Introduction

PolyDPR

1 minute read

Introduction

PALM

2 minute read

Introduction

Compressing Sentence Representation for Semantic Retrieval via Homomorphic Projective Distillation

2 minute read

Introduction

Task-guided Disentangled Tuning for Pretrained Language Models

1 minute read

Introduction

SCD: Self-Contrastive Decorrelation for Sentence Embeddings

2 minute read

Introduction

elBERto: Self-supervised Commonsense Learning for Question Answering

2 minute read

Introduction

AdaPrompt: Adaptive Model Training for Prompt-based NLP

1 minute read

Introduction

Mar 20 Research Wrapup

less than 1 minute read

Paper Summaries:

GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models

1 minute read

Introduction

Neural reality of argument structure constructions

2 minute read

Introduction

Back to Top ↑

Reading

ITERATED DECOMPOSITION: IMPROVING SCIENCE Q&A

8 minute read

Type: Paper

Scattered or Connected? An Optimized Parameter-efficient

1 minute read

Type: Paper

Multi-View Document Representation Learning for Open-Domain

1 minute read

Type: Paper

Combining Compressions for Multiplicative Size Scaling on Natural

1 minute read

Type: Paper

CHAPTERBREAK: A Challenge Dataset for Long Range Language Models

2 minute read

Type: Paper

LayoutLM: Pre-training of Text and Layout for Document Image Understanding

1 minute read

Introduction

STRUCTURE AND SEMANTICS PRESERVING DOCUMENT Representations

1 minute read

Introduction

GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation

2 minute read

Type: Paper

Back to Top ↑

NLP

Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words

2 minute read

Introduction

Capturing Failures of Large Language Models via Human Cognitive Biases

3 minute read

Intro

T5: Exploring the limits of Transfer Learning with a Unified Text to Text Transformer

4 minute read

Introduction

TransformerXL

2 minute read

Introduction Transformer models typically have a fixed context window that is hard to scale due to the $O(n^2)$ cost of the attention mechanism. Extending th...

Back to Top ↑

Transformer

SPANBERT

less than 1 minute read

What is the name of the SpanBERT paper? SpanBERT: Improving Pre-training by Representing and Predicting Spans Mandar Joshi, ...

Deep Continuous Prompt for Contrastive Learning of Sentence Embeddings

2 minute read

Introduction

TransformerXL

2 minute read

Introduction Transformer models typically have a fixed context window that is hard to scale due to the $O(n^2)$ cost of the attention mechanism. Extending th...

Back to Top ↑

Parameter Efficiency

Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Models

3 minute read

Introduction

Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models

2 minute read

Introduction

Back to Top ↑

xgboost

XGboost Part 1: Gradient Boosting

4 minute read

Introduction Xgboost is a powerful yet simple algorithm that has achieved state of the art results on tabular datasets. The Xgboost algorithm uses an ensembl...

Back to Top ↑

computer_vision

Parallelizing OpenCV For Real Time Object Detection

3 minute read

Introduction

Back to Top ↑

parallelization

Parallelizing OpenCV For Real Time Object Detection

3 minute read

Introduction

Back to Top ↑

audio

Wav2Vec part 1: Wav2Vec and VQwav2vec

2 minute read

What is Wav2Vec?

Back to Top ↑

self_supervised_learning

Wav2Vec part 1: Wav2Vec and VQwav2vec

2 minute read

What is Wav2Vec?

Back to Top ↑

parameter_efficiency

Compacter

2 minute read

Intro

Back to Top ↑

adapters

Compacter

2 minute read

Intro

Back to Top ↑

Transformers

Capturing Failures of Large Language Models via Human Cognitive Biases

3 minute read

Intro

Back to Top ↑

finetuning

NoisyTune: A Little Noise Can Help You Finetune

1 minute read

Type: Paper

Back to Top ↑

SentenceEmbeddings

PromptBERT improving BERT sentence embeddings with prompts

3 minute read

Intro

Back to Top ↑

Prompting

PromptBERT improving BERT sentence embeddings with prompts

3 minute read

Intro

Back to Top ↑

Few Shot Learning

Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models

2 minute read

Introduction

Back to Top ↑

Soft Prompting

Deep Continuous Prompt for Contrastive Learning of Sentence Embeddings

2 minute read

Introduction

Back to Top ↑

Paper Summaries

Mar 18 Research Wrapup

1 minute read

Quick hit thoughts on trending papers?

Back to Top ↑