Cross-Domain Pretraining for Multilingual Language Models
Pre-pretraining GPT-2 models on symbolic non-language data like music, probabilistic grammars, and cellular automata before multilingual BabyLM...

Created by Yifeng Peng
Daily curated AI papers with reproducible code, benchmarks, and practical industry takeaways
Explore the latest content tracked by AI Research Pulse
Pre-pretraining GPT-2 models on symbolic non-language data like music, probabilistic grammars, and cellular automata before multilingual BabyLM...
Solving ARC-AGI-3 shows engineering skill more than general intelligence.
AVATAR introduces a reinforcement learning approach that integrates seeing, hearing, and reasoning for agent alignment through two core components. The framework targets multimodal perception and alignment tasks relevant to deployed agents.
AI transforms materials R&D by creating a closed loop that converts data into candidate structures via prediction, simulation, and generative models...
Frontier lab employees voice high extinction risks yet cite no concrete, verifiable safety measures.
DRL schedulers succeed when state, action, and reward formulations directly match specific manufacturing constraints rather than generic templates.
-...
A novel sparse estimation approach constructs neural networks that explicitly harmonize predictive performance with guaranteed explainability.
Chinese-language research on trustworthy AI remains largely absent from mainstream debates despite China's major role in global development. Analysis...
HF's Sept 10 announcement creates an Open Alignment team for open-weight safety, alignment research, and cyber defense, yet no dedicated repository...
When AI generates proofs faster than mathematicians can build intuition and ownership, the field faces a core misalignment: solutions exist but human...
T1, a 122B MoE model, trains directly in real cloud shells for 300+ tool calls per task, using verifiers for rewards rather than sparse pass/fail...
A hybrid architecture uses deep encoding networks to extract low-dimensional state embeddings from joystick data and meta-reinforcement learning for...
Existing benchmarks collapse novelty into one score, hiding which dimension a model misjudges.
Medical LLM research faces a widening evaluation gap: lag between newest named model and publication grew from 1.33 to 6.08 quarters, while only 2.5%...
X-AuT progressively prunes audio-encoder layers in speech LLMs via cross-scale distillation and LoRA adaptation while keeping the backbone frozen,...
Selective knowledge distillation delivers a lightweight student model that preserves occlusion-robust features from a heavier teacher while hitting...
GLIE regenerates full late-interaction embeddings from just k=4 vectors per page, retaining ~80% of uncompressed nDCG@5 on ViDoRe v1—outperforming...
Skild S1 claims single-video task learning without retraining, yet long-horizon reliability and safety certification remain open questions.