abduldieter728
/
ichikawa-g
1
0
Code Issues Projects Wiki

Update 'DeepSeek-R1, at the Cusp of An Open Revolution'

master
Abdul Dieter 3 months ago
parent
3fc2c4cecf
commit
4e721d8a48
1 changed files with 40 additions and 0 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 40
      DeepSeek-R1%2C-at-the-Cusp-of-An-Open-Revolution.md

40
DeepSeek-R1%2C-at-the-Cusp-of-An-Open-Revolution.md
Unescape

@ -0,0 +1,40 @@ @@ -0,0 +1,40 @@
<br>[DeepSeek](http://log.tkj.jp) R1, the new entrant to the Large Language Model wars has developed quite a splash over the last few weeks. Its entrance into an area [controlled](https://rauszeit.blog) by the Big Corps, while pursuing asymmetric and [wiki.rolandradio.net](https://wiki.rolandradio.net/index.php?title=User:JosephY7659) unique techniques has been a [revitalizing eye-opener](http://www.estetattoo.at).<br>
<br>GPT [AI](http://yic.co.kr) improvement was beginning to reveal indications of slowing down, and has been observed to be reaching a point of [diminishing returns](http://218.94.103.2181982) as it runs out of data and calculate needed to train, fine-tune significantly big designs. This has actually turned the focus towards constructing "reasoning" [designs](https://moncuri.cl) that are post-trained through support learning, strategies such as inference-time and [test-time scaling](https://suarabaru.id) and search algorithms to make the [designs](https://www.avisfaenza.it) appear to believe and [forum.altaycoins.com](http://forum.altaycoins.com/profile.php?id=1067714) reason better. OpenAI's o1-series models were the first to attain this effectively with its inference-time scaling and Chain-of-Thought [thinking](http://hatzikekzi.de).<br>
<br>[Intelligence](http://www.gbpdesign.co.uk) as an [emergent](https://sound.co.id) home of Reinforcement Learning (RL)<br>
<br>Reinforcement Learning (RL) has actually been successfully utilized in the past by [Google's DeepMind](https://cho.today) team to construct extremely intelligent and customized systems where intelligence is observed as an emerging property through [rewards-based training](http://freeflashgamesnow.com) method that yielded achievements like AlphaGo (see my post on it here - AlphaGo: a journey to device intuition).<br>
<br>[DeepMind](https://thietbixangdau.vn) went on to build a series of Alpha * projects that attained numerous significant accomplishments using RL:<br>
<br>AlphaGo, defeated the world champion Lee Seedol in the video game of Go
<br>AlphaZero, a generalized system that found out to [play video](https://www.4techsrl.com) games such as Chess, Shogi and Go without human input
<br>AlphaStar, attained high [efficiency](https://www.meetgr.com) in the complex real-time strategy game StarCraft II.
<br>AlphaFold, a tool for [anticipating protein](http://185.87.111.463000) [structures](https://www.entrepicos.com) which [considerably advanced](https://mailtube.co.uk) computational biology.
<br>AlphaCode, a model created to generate computer programs, performing competitively in coding challenges.
<br>AlphaDev, a system [developed](https://ecsusa.net) to find unique algorithms, especially enhancing sorting [algorithms](https://tatilmaceralari.com) beyond [human-derived methods](http://git.in.ahbd.net).
<br>
All of these systems attained proficiency in its own area through self-training/self-play and by optimizing and optimizing the cumulative benefit in time by connecting with its environment where intelligence was observed as an emergent property of the system.<br>
<br>RL imitates the procedure through which an infant would [discover](https://www.dodgeball.org.my) to walk, through trial, [mistake](http://124.221.76.2813000) and first principles.<br>
<br>R1 design training pipeline<br>
<br>At a technical level, DeepSeek-R1 leverages a mix of Reinforcement Learning (RL) and [Supervised Fine-Tuning](https://africasfaces.com) (SFT) for its training pipeline:<br>
<br>Using RL and DeepSeek-v3, an interim reasoning design was built, called DeepSeek-R1-Zero, simply based upon RL without relying on SFT, which demonstrated remarkable reasoning abilities that matched the performance of OpenAI's o1 in certain criteria such as AIME 2024.<br>
<br>The model was however affected by [bad readability](https://www.meetgr.com) and language-mixing and is just an interim-reasoning design built on RL principles and self-evolution.<br>
<br>DeepSeek-R1-Zero was then utilized to create SFT information, which was [integrated](https://suarabaru.id) with monitored information from DeepSeek-v3 to re-train the DeepSeek-v3-Base design.<br>
<br>The new DeepSeek-v3-Base model then went through additional RL with prompts and circumstances to come up with the DeepSeek-R1 model.<br>
<br>The R1-model was then utilized to boil down a number of smaller open source models such as Llama-8b, Qwen-7b, 14b which outperformed larger models by a large margin, successfully making the smaller sized models more available and usable.<br>
<br>[Key contributions](https://uaslaboratory.synology.me) of DeepSeek-R1<br>
<br>1. RL without the need for SFT for emergent reasoning capabilities
<br>
R1 was the first open research study job to validate the effectiveness of [RL straight](https://git.citpb.ru) on the base design without depending on SFT as a [primary](https://www.andreottiroma.it) step, which led to the design establishing advanced reasoning abilities simply through self-reflection and [self-verification](http://120.25.206.2503000).<br>
<br>Although, it did deteriorate in its language abilities throughout the procedure, its Chain-of-Thought (CoT) abilities for [fixing intricate](https://www.jker.sg) issues was later used for [additional RL](https://www.dinodeangelis.com) on the DeepSeek-v3-Base model which became R1. This is a [substantial contribution](http://tashiro-s.weblike.jp) back to the research community.<br>
<br>The below analysis of DeepSeek-R1-Zero and OpenAI o1-0912 reveals that it is feasible to [attain robust](http://47.104.65.21419206) reasoning capabilities purely through RL alone, which can be more augmented with other methods to deliver even much better thinking efficiency.<br>
<br>Its quite intriguing, that the [application](http://lalcoradiari.com) of RL generates seemingly human abilities of "reflection", and coming to "aha" moments, triggering it to stop briefly, consider and concentrate on a specific aspect of the problem, leading to emerging capabilities to [problem-solve](http://biblbel.ru) as humans do.<br>
<br>1. Model distillation
<br>
DeepSeek-R1 also showed that bigger models can be distilled into smaller [designs](https://rarelypureneversimple.com) that makes innovative abilities available to resource-constrained environments, such as your laptop. While its not possible to run a 671b model on a stock laptop, you can still run a [distilled](https://www.ketaminaj.com) 14b model that is distilled from the bigger design which still [carries](http://zhangsheng1993.tpddns.cn3000) out much better than the majority of openly available designs out there. This allows intelligence to be more detailed to the edge, to [enable faster](https://shimashimashimatch619.com) [reasoning](http://dreamfieldkorea.com) at the point of experience (such as on a smart device, or on a [Raspberry](http://47.120.57.2263000) Pi), which paves method for more use cases and possibilities for [innovation](http://tongdaicu.com).<br>
<br>Distilled designs are [extremely](https://www.memorygrovemv.com) various to R1, which is a huge model with an entirely various design architecture than the [distilled](https://artistrybyhollylyn.com) variations, therefore are not straight equivalent in regards to capability, however are rather [constructed](http://124.222.84.2063000) to be more smaller sized and effective for more constrained environments. This strategy of having the [ability](https://www.finceptives.com) to distill a larger model's [capabilities](http://kiryu.deci.jp) to a smaller sized design for portability, availability, speed, and [expense](https://771xeon.ru) will bring about a lot of possibilities for using expert system in places where it would have otherwise not been possible. This is another essential contribution of this technology from DeepSeek, which I believe has even further capacity for democratization and availability of [AI](https://alandlous.com).<br>
<br>Why is this moment so significant?<br>
<br>DeepSeek-R1 was a [critical contribution](https://999vv.xyz) in many methods.<br>
<br>1. The contributions to the advanced and the open research helps move the field forward where everybody benefits, not just a couple of [extremely moneyed](https://emwritingsummer22.wp.txstate.edu) [AI](https://www.arovo.lu) [labs constructing](http://drinkoneforone.com) the next billion dollar design.
<br>2. Open-sourcing and making the design easily available follows an asymmetric strategy to the prevailing closed nature of much of the model-sphere of the larger gamers. [DeepSeek](https://mahenda.blog.binusian.org) ought to be applauded for making their contributions complimentary and open.
<br>3. It reminds us that its not just a [one-horse](https://firemuzik.com) race, and it incentivizes competitors, which has currently led to OpenAI o3-mini a cost-effective reasoning model which now shows the Chain-of-Thought reasoning. Competition is a great thing.
<br>4. We stand at the cusp of an explosion of [small-models](http://corex-shidai.com) that are hyper-specialized, and optimized for a specific use case that can be trained and [released cheaply](https://planetdump.com) for [resolving](http://47.120.57.2263000) issues at the edge. It raises a lot of exciting possibilities and is why DeepSeek-R1 is among the most turning points of [tech history](https://www.shapiropertnoy.com).
<br>
Truly exciting times. What will you develop?<br>
Write Preview
Loading…
Cancel
Save