åºæè¡šçŸæœåº (NER): åŸæ¥ã®èªç¶èšèªåŠçããAIé§åã®ããŒã¿æœåºãžã®é²å
åºæè¡šçŸæœåºïŒNamed Entity Recognition: NERïŒã¯ãèªç¶èšèªåŠçïŒNLPïŒã®åºç€æè¡ã®äžã€ã§ããããã¯ãããã¹ãããŒã¿ãªã©ã®éæ§é åããŒã¿ããã人åãçµç¹åãå°åãæ¥ä»ãéé衚çŸã補ååãšãã£ãç¹å®ã®ã«ããŽãªã«è©²åœããéèŠãªèŠçŽ ãèªåçã«ç¹å®ããåé¡ããããã»ã¹ã§ãã
NERããªããã°ãæ€çŽ¢ãšã³ãžã³ãã¬ã³ã¡ã³ããŒã·ã§ã³ã·ã¹ãã ãèªåããã¥ã¡ã³ãåæã·ã¹ãã ãªã©ã¯ãããã¹ãå
ã®ã誰ãããäœãããã©ãã§ãããã€ãè¡ã£ãã®ããæ£ç¢ºã«çè§£ããããšãå°é£ã«ãªããŸãã
æ¬èšäºã§ã¯ãNERã®åºæ¬æŠå¿µãæè¡ã®é²åããã»ã¹ããããŠãªãçŸä»£ã®çæAIããšã³ãã£ãã£æœåºãå®å
šã«å€é©ããã®ãã«ã€ããŠè©³ãã解説ããŸãã
1. NERæè¡ã®é²åããã»ã¹ AIããŒã¹ã®NERããªãããã»ã©é©æ°çãªã®ããçè§£ããããã«ãé廿°å幎éã«ããããšã³ãã£ãã£æœåºæè¡ã®æ©ã¿ãæ¯ãè¿ããŸãããã
第1äžä»£ïŒã«ãŒã«ããŒã¹ããã³èŸæžããŒã¹ã®ã·ã¹ãã åæã®NERã¯ãæ£èŠè¡šçŸïŒregexïŒã人æã§ç®¡çãããèŸæžïŒgazetteerïŒã«äŸåããŠããŸããã
ä»çµã¿: æœåºå¯Ÿè±¡ã®åèªãå°åããŒã¿ããŒã¹ã«ååšããå Žåããé»è©±çªå·ã®ãããªãã¿ãŒã³ïŒäŸïŒ[3æ¡]-[3æ¡]-[4æ¡]ïŒã«äžèŽããå Žåã«æœåºãããŸãã éç: éåžžã«è匱ã§ããã¹ãã«ãã¹ãæ°ãããšã³ãã£ãã£ãæèã«ããæå³ã®éãã«å¯Ÿå¿ã§ããŸãããäŸãã°ããAppleããæç©ã®ããªã³ãŽããæããŠããã®ããITäŒæ¥ã®ãã¢ããã«ããæããŠããã®ããåºå¥ã§ããŸããã§ããã 第2äžä»£ïŒåŸæ¥ã®æ©æ¢°åŠç¿ (CRF & SVM) 2000幎代ã«å
¥ããšãæ¡ä»¶ä»ã確çå ŽïŒCRFïŒããµããŒããã¯ã¿ãŒãã·ã³ïŒSVMïŒãªã©ã®çµ±èšçæ©æ¢°åŠç¿ã¢ãã«ãäž»æµãšãªããŸããã
ä»çµã¿: éçºè
ãæåã§ç¹åŸŽéïŒäŸïŒæ¥é èŸãæ¥å°ŸèŸã倧æåã»å°æåã®ãã¿ãŒã³ãªã©ïŒãèšèšããã©ãã«ä»ããããèšç·ŽããŒã¿ãçšããŠããŒã¯ã³ããšã³ãã£ãã£ã®äžéšã§ãã確çãäºæž¬ããŸãã éç: 倧éã®ã©ãã«ä»ãããŒã¿ãå¿
èŠã§ãããæåã§ã®ç¹åŸŽéèšèšã«ã¯èšå€§ãªåŽåãããããŸããã 第3äžä»£ïŒãã£ãŒãã©ãŒãã³ã° (BiLSTM-CRF & BERT) ãã£ãŒãã©ãŒãã³ã°ã®å°é ã«äŒŽããåæ¹åLSTMïŒBiLSTMïŒãšCRFãçµã¿åãããã¢ãã«ãããã®åŸã®BERTã«ä»£è¡šãããTransformerã¢ãã«ãNLPã«é©åœããããããŸããã
ä»çµã¿: åèªåã蟌ã¿ïŒWord EmbeddingsïŒãã»ãã³ãã£ãã¯ãªæå³ãæããæ·±å±€ãã¥ãŒã©ã«ãããã¯ãŒã¯ãæèãçè§£ããŸããBERTããŒã¹ã®ã¢ãã«ã¯ãååŸã®æèãããApple launched a new iPhoneãã®äžã®ãAppleããçµç¹ãšããŠèå¥ã§ããããã«ãªããŸããã éç: äŸç¶ãšããŠãã¡ã€ã³åºæã®ããŒã¿ã»ããã«å¯Ÿããæåž«ããã®ãã¡ã€ã³ãã¥ãŒãã³ã°ãå¿
èŠã§ãããäºåã«å®çŸ©ãããŠããªãæ°ããã«ããŽãªã®ãšã³ãã£ãã£ãååŠç¿ãªãã§æœåºããããšã¯å°é£ã§ããã 第4äžä»£ïŒçæAIãšLLMããŒã¹ã®NER çŸåšãGeminiãGPT-4ãLlama 3ãªã©ã®å€§èŠæš¡èšèªã¢ãã«ïŒLLMsïŒã¯ãã»ãã³ãã£ãã¯ãªçè§£ãšæç€ºè¿œåŸèœåã«ãã£ãŠNERãåŠçããŸãã
ä»çµã¿: Zero-shotãŸãã¯Few-shotã®ããã³ãããšã³ãžãã¢ãªã³ã°ã䜿çšããŠããŠãŒã¶ãŒã¯ä»»æã®ãšã³ãã£ãã£ã¿ã€ããæå®ãããããJSONãªã©ã®æ§é åããŒã¿ãšããŠè¿åŽããããã«LLMãžæç€ºã§ããŸãã éžã°ããçç±: è€éãªæ§æãçè§£ããã¹ãã«ãã¹ã«å¯Ÿå¿ã§ããææ§ãªæèãæšè«ããéå§ã«ããã£ãŠã®èšç·ŽããŒã¿ãäžåå¿
èŠãšããŸããã 2. AIããŒã¹ã®NERãšåŸæ¥ã®NERã®æ¯èŒ ç¹åŸŽ åŸæ¥ã®NER (BERT / CRF) AIããŒã¹ã®NER (LLMs) å¿
èŠãªåŠç¿ããŒã¿ 倧éïŒæ°å以äžã®ã©ãã«ä»ãããŒã¿ïŒ äžèŠã極å°éïŒZero-shot / Few-shotïŒ æè»æ§ äœãïŒåŠç¿æžã¿ã®ã«ããŽãªã®ã¿æœåºïŒ 極ããŠé«ãïŒããã³ããå
ã§ä»»æã®ã«ããŽãªãå®çŸ©å¯èœïŒ æèçè§£ äžçšåºŠïŒå±æçãªã³ã³ããã¹ããŠã£ã³ããŠïŒ æ·±ãïŒããã¥ã¡ã³ãå
šäœã®ã³ã³ããã¹ããæå³ãçè§£ïŒ æªç¥èªïŒOOVïŒãžã®å¯Ÿå¿ èŠæïŒæªåŠç¿ã®åèªã«èŠæŠïŒ åŸæïŒã»ãã³ãã£ãã¯ãªæšè«ãå©çšïŒ åŠçé床ãšã³ã¹ã é«éã»å®äŸ¡ïŒå°åã®CPU/GPUã§ããŒã«ã«å®è¡å¯èœïŒ äœéã»æ¯èŒçé«ã³ã¹ãïŒå€§èŠæš¡ã¢ãã«ã®æšè«ãå¿
èŠïŒ 3. AIããŒã¹ã®NERã®äž»ãªå¿çšåé AIããŒã¹ã®åºæè¡šçŸæœåºã¯ãåãªãããã¹ãã®ãã€ã©ã€ãã«ãšã©ãŸããŸãããéæ§é åããã¹ããæ§é åãããã¢ã¯ã·ã§ã³å¯èœãªJSONããŒã¿ã«å€æããããšã§ã匷åãªèªååãå®çŸããŸãã
AI
åºæè¡šçŸæœåº
èªç¶èšèªåŠç
æ©æ¢°åŠç¿
å€§èŠæš¡èšèªã¢ãã«