Base Model for TransMLA
mengfanxu
fxmeng
AI & ML interests
None yet
Recent Activity
upvoted a paper 1 day ago
HISA: Efficient Hierarchical Indexing for Fine-Grained Sparse Attention authored a paper 3 days ago
LIFT: Improving Long Context Understanding of Large Language Models
through Long Input Fine-Tuning authored a paper 3 days ago
LIFT: Improving Long Context Understanding Through Long Input
Fine-TuningOrganizations
None yet
Principal Singular Values and Singular Vectors Adaptation
Principal Singular Values and Singular Vectors Adaptation
Substructure of mistralai/Mixtral-8x7B-Instruct-v0.1
Base Model for TransMLA
Principal Singular Values and Singular Vectors Adaptation
Principal Singular Values and Singular Vectors Adaptation
Principal Singular Values and Singular Vectors Adaptation
Principal Singular Values and Singular Vectors Adaptation
https://arxiv.org/abs/2404.02948
Substructure of mistralai/Mixtral-8x7B-Instruct-v0.1