うわぁ。
以前、下記を書いて、「まだまだ続くよ！」とか言ってましたが…
7月20日のことでした。(;^ω^)
もう4か月も経ってるやんけ！！！( ﾟДﾟ)
いやー、光陰矢の如し。7月末からいろんなことがあったなぁ…( ˊᵕˋ )
…と、ついつい楽しい夏の思い出を思い返しちゃいましたが、本題は、Alpha Go Zeroの論文の翻訳の最後を書くことでした。
仕事でもAIの開発を進めておりまして、少し詳しくなったと思います。
では、Go!
原文はこちら。
https://www.nature.com/articles/nature24270.epdf
前回の続きということで、付録的な、「METHODS」という部分の後半になります。
Domain knowledge というところからですね。ではGo!
Domain knowledge
Domain knowledgeとは、専門知識のことである。碁のプログラムでいえば、碁のうちかた、どうやって強い手を繰り出すか、というような専門知識のことである。
Alpha Go Zeroの一番の功績は、専門知識なくして人間を超える知力を達成したということにある。
この功績を明らかにすべく、Alpha Go Zeroが使っている専門知識をトレーニングプロシージャーやMCTS内にかかわらず、数え上げてみることにする。
これらは、ほかのゲーム（マルコフゲーム プレーヤーが1対1で進める手によって勝率などが左右される、オセロ・将棋・碁などのゲームの総称と思われ。）に使うときは、置き換えられるものである。
①ゲームのルール
Alpha Go Zeroはゲームのルールについては、完全に知っている。MCTSのポジションのシミュレーションや最終手を打った時のスコアをつけるのに使われている。
ゲームは両方のプレーヤーがパスしたときと722手進んだ時に終了する。また、プレーヤーはルールにのっとって、有効な手を打つ必要がある。
②スコアリング
Alpha Go ZeroではMCTSのシミュレーションとセルフプレイのトレーニングの最中にTromp-Taylorスコアリングを使っている。
なぜかというと、韓国式・日本式・中国式にせよ、人間のスコアリングはよく定義されてないからである。
ちなみにAlpha Go Zeroで採用しているのは中国式である。（この辺、私が囲碁わからないのでよくわからなかったです(;´･ω･)）
③盤面
盤面は、19×19の画像的なイメージで扱われる。ニューラルネットワークはこのボードに合わせた形になっている。
④盤面の回転など
碁のルールは盤面を回転しても反転しても盤面の強さは変わらない。
それを利用して、MCTSの途中で、盤面の回転と、反転状態もデータの中に入れることができている。
komiっていうものを抜かせば、碁は色を取り換えてもなりたつ。色を変えても成り立つことにより、盤面をどちらのプレーヤーから見るか、と言ったことが成り立たせられる。
以上が、Alpha Go Zeroが使っている専門知識の全部である。Alpha Go Zeroがディープニューラルネットワークを使っているのは、MCTSの末端のノードを評価するときと、手を選択するときだけ。
ロールアウトポリシーも使ってないし、ツリーポリシーも使っていないし、MCTSは他のヒューリスティックだったり専門知識によるルールによって評価されたりもしない。
ルール違反の手は使わない。プレーヤーの「目」を埋めることもしない。（多分、碁の話と思われ）
このアルゴリズムはランダムな初期パラメーター（ニューラルネットワークに入れるやつ）で始められた。
ニューラルネットワークの構造は、先進的な画像認識の構造である。ハイパーパラメーターは学習の中で変わっていく。
MCTSの探索パラメーターは1個前にトレーニングしたニューラルネットワークを使ったセルフプレイを最適化するために、ガウス過程最適化（でいいのかな？）を使って選択されていく。
より大きい実行のために（40ブロック、40日かける実行）MCTSの探索パラメータはより小さい実行でトレーニングされたニューラルネットワーク（20ブロック、3日）を使って再最適化される。
トレーニングのアルゴリズムは人間が介在することなく、自主的にアップデートされていくのである。
Self-play training pipeline
Alpha Go Zeroのセルフプレイのトレーニングパイプラインは3つに分かれていて、全部が非同期で平行に動作する。
ニューラルネットワークのパラメーターΘは継続的にセルフプレイのデータで最適化されていく。
Alpha Go Zeroのプレーヤーαは継続的に評価されていく。
ベストなプレーヤーはセルフプレイのデータを生み出すために使われていく。
今日はここまで~ ハァハァ(*´Д`)
Magnifficent goods fro you, man. Ihave understand your stuff previous to
and you’re jjust extremely great. I really
like what you have acquired here, really like what you’re stating and the way in which you say it.
You make it enjoyable and you still twke care of too
keep it smart. I can not wait to read far more from you. This is actually
a wonderful website.
Feel free to visit my blog post; Bursa escort
I appreciate the actionable takeaways provided at the end of each section. They serve as handy reminders of the key points discussed and make it easier to implement the insights shared in the blog post. | https://www.pinnedin.com/
Overall, this blog post was a valuable read packed with practical tips and thought-provoking insights. I’ll definitely be revisiting it as I continue to navigate my professional journey.
I appreciate the emphasis on mental health awareness in the workplace. It’s crucial to prioritize self-care, and this blog serves as a gentle reminder to check in with ourselves and our colleagues regularly. | https://www.pinnedin.com/
WordPress Eklenti
It was a perfect sharing mate. it helped me a lot cute
You really make it seem really easy together with your presentation but I
find this topic too be really one thing that I feel I might never understand.
It sort of fewls too complex and very large ffor me. I am having a look forward for your subsequent post, I wull try to get the dangle of it!
Feel free to surf to my web page … raylı branda fiyatları
My family all the time say that I am wasting myy time here at web, except I know I am getting experience
all the tome by reading such fasstidious articles or reviews.
Here is my website … Antalya Airport Transfers
Hi there, yeah this post is in fact fastidious and I
have learned lot of things from it about blogging.
thanks.
Also visit my page :: yes or no
After I initially commented I appar to have clicked on the -Notiy me whenn new comments are
added- checkbox and from nnow on every time a
comment is added I get four emails with the ezact same comment.
Therde has to be a means you are able to remove me from that service?
Appreciate it!
Also visit my blog Beyin Cerrahi
Appreciate this post. Will try it out.
Feeel free to surf too mmy site kalça protezi ameliyatı sonrası yorumlar
Iconstantly spent my half an hour tto read this webpage’s articles or reviews everyday along with a cup of coffee.
my web-site :: Depo çadırı satın al
It was a perfect sharing bro. very useful cute
Currently it looks like Drupal is the top blogging platform available right now.
(from what I’ve read) Is that what you’re using
on your blog?
Here is my website – Almanca yeminli tercüme
I think this is one of the so much vital info for me.
And i’m happy reading your article. But wanna statement on feww normal
things, The web site taste is perfect, thee
articles is really great : D. Excellent task, cheers
Check out my blog post Abdullah Yanık
It was a perfect sharing admin. it helped me a lot tahnxss
It was a perfect sharing bro. very useful thx
Would you look at the beauty of sharing? Man. very nice thing thanx
wonderful points altogether, you simply gained a logo new reader.
What might you recommend in regards to your postt
that you simply made a few days ago? Any certain?
My website – イスタンブールの毛髪移植の価格
Would you look at the beauty of sharing? brother. very nice thing cute
Would you look at the beauty of sharing? hey. very useful to me thank you
Would you look at the beauty of sharing? Man. very nice thing thank you
Do you mind if I quote a few of your posts as long as I provide
credit and sources back to your site? My blog is in the very
same niche as yours and my visitors would genuinely benefit from
a lot of the information you present here.
Please let me know if this okay with you. Thanks!
hello there and thank you for your information – I’ve certainly picked up something new from right here.
I did however expertise several technical issues using this website, as I
experienced to reload the site a lot of times previous tto I could get it too load properly.
I had been wondering if yourr web hosting is OK?
Not that I am complaining, but sluggish loadingg instances times will very frequently affect your placedment in google and could damage your quality score if ads and marketing with Adwords.
Well I’m adding this RSS to my e-mail and could look out for a lot
more of your respective fascinaing content. Ensure thhat you update this again soon.
Here is my website :: yeminli tercüme
Yazarın duyarlı yaklaşımı ve empatik bakış açısı beni derinden etkiledi. Bu tür içeriklerin paylaşılması gerçekten önemli. | Fatih Mahallesi, Tuzla Koltuk Yıkama
takipçi al
Would you look at the beauty of sharing? Man. very useful to me cute
Would you look at the beauty of sharing? hey. very nice thing thank you
Gri Melanj Bisiklet Yaka Oversize Crop Sweatshirt ve Paçası Lastikli Jogger Eşofman Takım | Your blog posts never fail to impress me. Reading them is a pleasure, thank you.
Greetings I am so thrilled I found yor blog, I really found
you by error, while I was researching oon Google
for something else, Nonetheless I am here noow and would just like to say thanks a lot for a remarkable post and
a all round interesting blog (I also love
the theme/design), I don’t have time to
go through it all aat the moment but I have book-marked it
and also added your RSS feeds, so when I have time I will be back to read more, Please do keep up the superb b.
My blog – Trasplante Capilar FUE Zafiro en Turquía
Rasimpaşa Beton Kesme | Rüzgar Karot’un profesyonel yaklaşımı ve etkili iletişimi sayesinde işlerimiz sorunsuz ilerledi.
Akdağmadeni / Yozgat Toptan Giyim | RENE Wholesale Textile and Clothing Solutions’ product range is impressively diverse. You can find products to suit any style or need.
Would you look at the beauty of sharing? brother. very nice thing thanx
Beşiktaş Market Poşet | Eminoğlu Ambalaj’ın çeşitli ambalaj seçenekleri, ihtiyaçlarımıza uygun çözümler sunuyor. Her zaman memnun kalıyoruz!
Would you look at the beauty of sharing? brother. very nice thing thank you
sana sokayım
Reutje | Deze blog heeft me echt geholpen meer te begrijpen over de waarde die ze creëren in webdesign en softwareontwikkeling. Ik ben onder de indruk van hun werk.
Wat is Monitor? | Ik ben altijd onder de indruk van de innovatieve oplossingen die MAFA biedt op het gebied van webdesign en softwareontwikkeling. Ze zijn echt pioniers in hun vakgebied.
Lapseki Toptan Giyim | The customer service at RENE Wholesale Textile and Clothing Solutions is exemplary. They’re always prompt and helpful.
Sineklik Sistemleri | Venster Systems’ screens not only keep flies out but also add a touch of sophistication to my home.
Bartın Jaluzi Sistemleri | Evimizin dekorasyonunu tamamlarken kaliteli ve uygun fiyatlı bir çözüm arıyordum, Venster Systems bu ihtiyacımı fazlasıyla karşıladı.
Wat is Settopbox? | MAFA’s Ruf im Bereich Webdesign und Softwareentwicklung spricht für sich. Ich kann es kaum erwarten, mit ihnen an meinem nächsten Projekt zu arbeiten.
Hey there! Someone in my Myspace group shared this website with
us so I came to take a look. I’m definitely enjoying tthe
information. I’m bookmarking and will be tweetinng this to my followers!
Terrific blog and terrific style and design.
Feel free to visit my page; random wheel spinner
eski karim duyguyla sikis sohbete hemen katil sohbet.net
Cobblers Recipes | Your blog is a lighthouse guiding lost souls through the stormy seas of life. Thank you for the direction.
I believe everything posted was very logical. But, think on this, suppose
you tyoed a catchier title? I am not saying yopur information is noot good, but what if yyou added a
post title that makes people desire more? I mean Alpha Go Zeroの論文の翻訳 その5 – Summer Snow is
a little vanilla. You cokuld look aat Yahoo’s home page and watch how they
create post headlines to grab viewers too click.
You might try adding a video oor a pic or two tto grab readers
interested about what you’ve written. Trapianto di capelli Dhi in Turchia my opinion, iit would bring youjr posts a little bit more interesting.
Would you look at the beauty of sharing? brother. it helped me a lot thanx
Spot on with this write-up, I really believe this web site needs far more attention.
I’ll probably be returning to see more, thanks for the info!
My web page … 30 faktör Güneş kremi
Thanks for your marvelous posting! I definitely enjoyed reading it, you happen to be a great author.I wil always
bookmark your bpog annd will eventually comee
back later on. I want to encourage you to continue your great work, have a nice holiday weekend!
Check oout my web site; dumanbet
Bobcat Warning Lights, Symbols and Meanings