CoFrGeNets replace the ‘bones’ of transformer-based models

Details

IBM Research introduced Continued Fraction Generative Networks (CoFrGeNets), a new model architecture that replaces the two dominant components of transformer-based models — multi-head attention and feed-forward networks — with structures derived from continued fractions. A continued fraction expresses complex functions as nested sequences of divisions, allowing compactly represented "ladders" of computation where each layer depends on the prior layer's reciprocal. The result, according to the team led by principal research scientist Amit Dhurandhar, is higher representational complexity without proportional parameter growth. The work was presented at ICML 2026 in Seoul, a top-tier machine learning venue that provides meaningful peer-review credibility. The project extends IBM's earlier research applying continued fractions to classification tasks, where they showed advantages over conventional architectures in controlled settings. Moving to generative models is described as a significant architectural jump, but the blog post does not include specific benchmark scores, parameter counts, or model sizes — making it impossible to independently assess the "competitive and in many cases even better" claim without reading the full ICML paper. Transformer alternatives face a historically difficult adoption curve despite theoretical advantages. If CoFrGeNets' efficiency gains hold at larger scales, the implications for training cost and inference energy consumption are meaningful — but that validation remains ahead.