AI scoring for international large-scale assessments using a deep learning model and multilingual data:
Gespeichert in:
| Beteilige Person: | |
|---|---|
| Weitere beteiligte Personen: | , , , , |
| Format: | Elektronisch E-Book |
| Sprache: | Englisch |
| Veröffentlicht: |
Paris
OECD Publishing
2023
|
| Schriftenreihe: | OECD Education Working Papers
no.287 |
| Schlagwörter: | |
| Links: | https://doi.org/10.1787/9918e1fb-en |
| Abstract: | Artificial Intelligence (AI) scoring for constructed-response items, using recent advancements in multilingual, deep learning techniques utilising models pre-trained with a massive multilingual text corpus, is examined using international large-scale assessment data. Historical student responses to Reading and Science literacy cognitive items developed under the PISA analytical framework are used as training data for deep learning together with multilingual data to construct an AI model. The trained AI models are then used to score and the results compared with human-scored data. The score distributions estimated based on the AI-scored data and the human-scored data are highly consistent with each other; furthermore, even item-level psychometric properties of the majority of items showed high levels of agreement, although a few items showed discrepancies. This study demonstrates a practical procedure for using a multilingual data approach, and this new AI-scoring methodology reached a practical level of quality, even in the context of an international large-scale assessment |
| Umfang: | 1 Online-Ressource (34 Seiten) |
| DOI: | 10.1787/9918e1fb-en |
Internformat
MARC
| LEADER | 00000nam a22000001cb4500 | ||
|---|---|---|---|
| 001 | BV049038321 | ||
| 003 | DE-604 | ||
| 007 | cr|uuu---uuuuu | ||
| 008 | 230706s2023 xx o|||| 00||| eng d | ||
| 024 | 7 | |a 10.1787/9918e1fb-en |2 doi | |
| 035 | |a (ZDB-13-SOC)092691269 | ||
| 035 | |a (OCoLC)1390808280 | ||
| 035 | |a (DE-599)KEP092691269 | ||
| 040 | |a DE-604 |b ger |e rda | ||
| 041 | 0 | |a eng | |
| 049 | |a DE-521 |a DE-1028 |a DE-573 |a DE-92 |a DE-898 |a DE-1049 |a DE-861 |a DE-91 |a DE-384 |a DE-473 |a DE-355 |a DE-20 |a DE-824 |a DE-29 |a DE-739 |a DE-188 |a DE-19 | ||
| 100 | 1 | |a Okubo, Tomoya |e Verfasser |4 aut | |
| 245 | 1 | 0 | |a AI scoring for international large-scale assessments using a deep learning model and multilingual data |c Tomoya, Okubo ... [et al] |
| 264 | 1 | |a Paris |b OECD Publishing |c 2023 | |
| 300 | |a 1 Online-Ressource (34 Seiten) | ||
| 336 | |b txt |2 rdacontent | ||
| 337 | |b c |2 rdamedia | ||
| 338 | |b cr |2 rdacarrier | ||
| 490 | 0 | |a OECD Education Working Papers |v no.287 | |
| 520 | 3 | |a Artificial Intelligence (AI) scoring for constructed-response items, using recent advancements in multilingual, deep learning techniques utilising models pre-trained with a massive multilingual text corpus, is examined using international large-scale assessment data. Historical student responses to Reading and Science literacy cognitive items developed under the PISA analytical framework are used as training data for deep learning together with multilingual data to construct an AI model. The trained AI models are then used to score and the results compared with human-scored data. The score distributions estimated based on the AI-scored data and the human-scored data are highly consistent with each other; furthermore, even item-level psychometric properties of the majority of items showed high levels of agreement, although a few items showed discrepancies. This study demonstrates a practical procedure for using a multilingual data approach, and this new AI-scoring methodology reached a practical level of quality, even in the context of an international large-scale assessment | |
| 650 | 4 | |a Education | |
| 650 | 4 | |a Science and Technology | |
| 700 | 1 | |a Houlden, Wayne |4 ctb | |
| 700 | 1 | |a Montuoro, Paul |4 ctb | |
| 700 | 1 | |a Reinertsen, Nate |4 ctb | |
| 700 | 1 | |a Tse, Chi Sum |4 ctb | |
| 700 | 1 | |a Bastianic, Tanja |4 ctb | |
| 856 | 4 | 0 | |u https://doi.org/10.1787/9918e1fb-en |x Verlag |z kostenfrei |3 Volltext |
| 912 | |a ZDB-13-SOC | ||
| 943 | 1 | |a oai:aleph.bib-bvb.de:BVB01-034300923 | |
Datensatz im Suchindex
| DE-BY-TUM_katkey | 2751109 |
|---|---|
| _version_ | 1822630110054842369 |
| adam_text | |
| any_adam_object | |
| author | Okubo, Tomoya |
| author2 | Houlden, Wayne Montuoro, Paul Reinertsen, Nate Tse, Chi Sum Bastianic, Tanja |
| author2_role | ctb ctb ctb ctb ctb |
| author2_variant | w h wh p m pm n r nr c s t cs cst t b tb |
| author_facet | Okubo, Tomoya Houlden, Wayne Montuoro, Paul Reinertsen, Nate Tse, Chi Sum Bastianic, Tanja |
| author_role | aut |
| author_sort | Okubo, Tomoya |
| author_variant | t o to |
| building | Verbundindex |
| bvnumber | BV049038321 |
| collection | ZDB-13-SOC |
| ctrlnum | (ZDB-13-SOC)092691269 (OCoLC)1390808280 (DE-599)KEP092691269 |
| discipline | Wirtschaftswissenschaften |
| doi_str_mv | 10.1787/9918e1fb-en |
| format | Electronic eBook |
| fullrecord | <?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>00000nam a22000001cb4500</leader><controlfield tag="001">BV049038321</controlfield><controlfield tag="003">DE-604</controlfield><controlfield tag="007">cr|uuu---uuuuu</controlfield><controlfield tag="008">230706s2023 xx o|||| 00||| eng d</controlfield><datafield tag="024" ind1="7" ind2=" "><subfield code="a">10.1787/9918e1fb-en</subfield><subfield code="2">doi</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(ZDB-13-SOC)092691269</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(OCoLC)1390808280</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)KEP092691269</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-604</subfield><subfield code="b">ger</subfield><subfield code="e">rda</subfield></datafield><datafield tag="041" ind1="0" ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="049" ind1=" " ind2=" "><subfield code="a">DE-521</subfield><subfield code="a">DE-1028</subfield><subfield code="a">DE-573</subfield><subfield code="a">DE-92</subfield><subfield code="a">DE-898</subfield><subfield code="a">DE-1049</subfield><subfield code="a">DE-861</subfield><subfield code="a">DE-91</subfield><subfield code="a">DE-384</subfield><subfield code="a">DE-473</subfield><subfield code="a">DE-355</subfield><subfield code="a">DE-20</subfield><subfield code="a">DE-824</subfield><subfield code="a">DE-29</subfield><subfield code="a">DE-739</subfield><subfield code="a">DE-188</subfield><subfield code="a">DE-19</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Okubo, Tomoya</subfield><subfield code="e">Verfasser</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">AI scoring for international large-scale assessments using a deep learning model and multilingual data</subfield><subfield code="c">Tomoya, Okubo ... [et al]</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="a">Paris</subfield><subfield code="b">OECD Publishing</subfield><subfield code="c">2023</subfield></datafield><datafield tag="300" ind1=" " ind2=" "><subfield code="a">1 Online-Ressource (34 Seiten)</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="b">c</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="b">cr</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="490" ind1="0" ind2=" "><subfield code="a">OECD Education Working Papers</subfield><subfield code="v">no.287</subfield></datafield><datafield tag="520" ind1="3" ind2=" "><subfield code="a">Artificial Intelligence (AI) scoring for constructed-response items, using recent advancements in multilingual, deep learning techniques utilising models pre-trained with a massive multilingual text corpus, is examined using international large-scale assessment data. Historical student responses to Reading and Science literacy cognitive items developed under the PISA analytical framework are used as training data for deep learning together with multilingual data to construct an AI model. The trained AI models are then used to score and the results compared with human-scored data. The score distributions estimated based on the AI-scored data and the human-scored data are highly consistent with each other; furthermore, even item-level psychometric properties of the majority of items showed high levels of agreement, although a few items showed discrepancies. This study demonstrates a practical procedure for using a multilingual data approach, and this new AI-scoring methodology reached a practical level of quality, even in the context of an international large-scale assessment</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Education</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Science and Technology</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Houlden, Wayne</subfield><subfield code="4">ctb</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Montuoro, Paul</subfield><subfield code="4">ctb</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Reinertsen, Nate</subfield><subfield code="4">ctb</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Tse, Chi Sum</subfield><subfield code="4">ctb</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Bastianic, Tanja</subfield><subfield code="4">ctb</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://doi.org/10.1787/9918e1fb-en</subfield><subfield code="x">Verlag</subfield><subfield code="z">kostenfrei</subfield><subfield code="3">Volltext</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">ZDB-13-SOC</subfield></datafield><datafield tag="943" ind1="1" ind2=" "><subfield code="a">oai:aleph.bib-bvb.de:BVB01-034300923</subfield></datafield></record></collection> |
| id | DE-604.BV049038321 |
| illustrated | Not Illustrated |
| indexdate | 2025-01-29T19:03:19Z |
| institution | BVB |
| language | English |
| oai_aleph_id | oai:aleph.bib-bvb.de:BVB01-034300923 |
| oclc_num | 1390808280 |
| open_access_boolean | 1 |
| owner | DE-521 DE-1028 DE-573 DE-92 DE-898 DE-BY-UBR DE-1049 DE-861 DE-91 DE-BY-TUM DE-384 DE-473 DE-BY-UBG DE-355 DE-BY-UBR DE-20 DE-824 DE-29 DE-739 DE-188 DE-19 DE-BY-UBM |
| owner_facet | DE-521 DE-1028 DE-573 DE-92 DE-898 DE-BY-UBR DE-1049 DE-861 DE-91 DE-BY-TUM DE-384 DE-473 DE-BY-UBG DE-355 DE-BY-UBR DE-20 DE-824 DE-29 DE-739 DE-188 DE-19 DE-BY-UBM |
| physical | 1 Online-Ressource (34 Seiten) |
| psigel | ZDB-13-SOC |
| publishDate | 2023 |
| publishDateSearch | 2023 |
| publishDateSort | 2023 |
| publisher | OECD Publishing |
| record_format | marc |
| series2 | OECD Education Working Papers |
| spellingShingle | Okubo, Tomoya AI scoring for international large-scale assessments using a deep learning model and multilingual data Education Science and Technology |
| title | AI scoring for international large-scale assessments using a deep learning model and multilingual data |
| title_auth | AI scoring for international large-scale assessments using a deep learning model and multilingual data |
| title_exact_search | AI scoring for international large-scale assessments using a deep learning model and multilingual data |
| title_full | AI scoring for international large-scale assessments using a deep learning model and multilingual data Tomoya, Okubo ... [et al] |
| title_fullStr | AI scoring for international large-scale assessments using a deep learning model and multilingual data Tomoya, Okubo ... [et al] |
| title_full_unstemmed | AI scoring for international large-scale assessments using a deep learning model and multilingual data Tomoya, Okubo ... [et al] |
| title_short | AI scoring for international large-scale assessments using a deep learning model and multilingual data |
| title_sort | ai scoring for international large scale assessments using a deep learning model and multilingual data |
| topic | Education Science and Technology |
| topic_facet | Education Science and Technology |
| url | https://doi.org/10.1787/9918e1fb-en |
| work_keys_str_mv | AT okubotomoya aiscoringforinternationallargescaleassessmentsusingadeeplearningmodelandmultilingualdata AT houldenwayne aiscoringforinternationallargescaleassessmentsusingadeeplearningmodelandmultilingualdata AT montuoropaul aiscoringforinternationallargescaleassessmentsusingadeeplearningmodelandmultilingualdata AT reinertsennate aiscoringforinternationallargescaleassessmentsusingadeeplearningmodelandmultilingualdata AT tsechisum aiscoringforinternationallargescaleassessmentsusingadeeplearningmodelandmultilingualdata AT bastianictanja aiscoringforinternationallargescaleassessmentsusingadeeplearningmodelandmultilingualdata |