Perplexity Scaling with Context Length(Lower perplexity is better)2k4k8k16k32k64k128kContext Length (tokens)181614121086PerplexityPlateau at 16kTransformerMambaTTT-Linear