Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
zuzun
on May 14, 2023
|
parent
|
context
|
favorite
| on:
Attention with Linear Biases (ALiBi)
If I understand it correctly, you are only attending preceding tokens in your paper. Can the constant bias matrix be made symmetric for unmasked tasks?
Consider applying for YC's Summer 2026 batch! Applications are open till May 4
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
