1. Home
  2. Vol 8, No 5 (2024)
  3. Jannuzzi

Zero-Shot Prompting Strategies for Table Question Answering with a Low-Resource Language

Marcelo Jannuzzi, Yuriy Perezhohin, Fernando Peres, Mauro Castelli, Aleš Popovič

Abstract


This work explores the application of zero-shot prompting strategies for table question answering (TQA) in Portuguese, focusing specifically on the Text2SQL task. This task involves translating questions posed in natural language into Structured Query Language (SQL) queries, which can be executed against a database to answer the original question. Given the popularity of relational databases across various domains, advancements in this field can substantially impact the accessibility and democratization of data as simpler and more intuitive interfaces for database interaction are developed. Despite this significant potential, progress in developing Portuguese TQA solutions remains limited. The proposed approach leverages Large Language Models (LLMs)—specifically the GPT-3.5 and GPT-4 models—through zero-shot prompting. The primary objectives are to assess the effectiveness of such LLMs in this task and to identify the most suitable prompt styles. These are evaluated using a Portuguese translation of the popular Spider Text2SQL benchmark. Results reveal that the proposed approach can generate adequate SQL queries to answer Portuguese language questions about various databases, mainly when using GPT-4. The findings suggest that including schema information and database content in the prompts is critical for satisfactory outcomes.

 

Doi: 10.28991/ESJ-2024-08-05-020

Full Text: PDF


Keywords


Text to SQL; Natural Language Processing; Computational Linguistics; Zero-shot Prompting.

References


Price, P. J. (1990). Evaluation of spoken language systems. Proceedings of the Workshop on Speech and Natural Language - HLT ’90, 91–95. doi:10.3115/116580.116612.

Zelle, J. M., & Mooney, R. J. (1996). Learning to parse database queries using inductive logic programming. Proceedings of the National Conference on Artificial Intelligence, 4-8 August, 1996, Portland, United States.

Li, F., & Jagadish, H. V. (2014). Constructing an interactive natural language interface for relational databases. Proceedings of the VLDB Endowment, 8(1), 73–84. doi:10.14778/2735461.2735468.

Saha, D., Floratou, A., Sankaranarayanan, K., Minhas, U. F., Mittal, A. R., & Özcan, F. (2016). ATHENA: An ontologydriven system for natural language querying over relational data stores. Proceedings of the VLDB Endowment, 9(12), 1209–1220. doi:10.14778/2994509.2994536.

Vaswani, A. (2017). Attention is all you need. Advances in Neural Information Processing Systems. 31st Conference on Neural Information Processing Systems, 4-9 December, 2017, Long Beach, United States.

Wang, B., Shin, R., Liu, X., Polozov, O., & Richardson, M. (2020). RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 1-12. doi:10.18653/v1/2020.acl-main.677.

Scholak, T., Schucher, N., & Bahdanau, D. (2021). PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 1-7. doi:10.18653/v1/2021.emnlp-main.779.

Rajkumar, N., Li, R., & Bahdanau, D. (2022). Evaluating the Text-to-SQL Capabilities of Large Language Models. arXiv Preprint, arXiv:2204.00498. doi:10.48550/arXiv.2204.00498.

Brown, T. B. (2020). Language models are few-shot learners. arXiv Preprint, arXiv:2005.14165.doi:10.48550/arXiv.2005.14165.

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., ... & Lowe, R. (2022). Training language models to follow instructions with human feedback. Advances in neural information processing systems, MIT Press, Cambridge, United States. doi:10.48550/arXiv.2203.02155.

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., ... & McGrew, B. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774. doi:10.48550/arXiv.2303.08774.

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., ... & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140), 1-67.

Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., ... & Bari, M. S. (2022). Bloom: A 176b-parameter open-access multilingual language model. arXiv Preprint, arXiv:2211.05100. doi:10.48550/arXiv.2211.05100.

Sun, Y., Wang, S., Feng, S., Ding, S., Pang, C., Shang, J., ... & Wang, H. (2021). Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv preprint arXiv:2107.02137.. doi:10.48550/arXiv.2107.02137.

da Silva, C.F.M., Jindal, R. (2021). SQL Query from Portuguese Language Using Natural Language Processing. In: Garg, D., Wong, K., Sarangapani, J., Gupta, S.K. (eds) Advanced Computing. IACC 2020. Communications in Computer and Information Science. Springer, Singapore. doi:10.1007/978-981-16-0401-0_25.

José, M. A., & Cozman, F. G. (2021). mRAT-SQL+GAP: A Portuguese Text-to-SQL Transformer. In: Britto, A., Valdivia Delgado, K. (eds) Intelligent Systems. BRACIS 2021. Lecture Notes in Computer Science. Springer, Cham, Switzerland. doi:10.1007/978-3-030-91699-2_35.

Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., ... & Radev, D. (2018). Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint, arXiv:1809.08887.. doi:10.48550/arXiv.1809.08887.

Jin, N., Siebert, J., Li, D., Chen, Q. (2022). A Survey on Table Question Answering: Recent Advances. Knowledge Graph and Semantic Computing: Knowledge Graph Empowers the Digital Economy. Communications in Computer and Information Science. Springer, Singapore. doi:10.1007/978-981-19-7596-7_14.

Pasupat, P., & Liang, P. (2015). Compositional Semantic Parsing on Semi-Structured Tables. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. doi:10.3115/v1/p15-1142.

Misra, D., Chang, M.-W., He, X., & Yih, W. (2018). Policy Shaping and Generalized Update Equations for Semantic Parsing from Denotations. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. doi:10.18653/v1/d18-1266.

Zhong, V., Xiong, C., & Socher, R. (2017). Seq2sql: Generating structured queries from natural language using reinforcement learning. doi:10.48550/arXiv.1709.00103.

Sun, Y., Tang, D., Duan, N., Ji, J., Cao, G., Feng, X., Qin, B., Liu, T., & Zhou, M. (2018). Semantic Parsing with Syntax- and Table-Aware SQL Generation. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. doi:10.18653/v1/p18-1034.

Nan, L., Hsieh, C., Mao, Z., Lin, X. V., Verma, N., Zhang, R., Kryściński, W., Schoelkopf, H., Kong, R., Tang, X., Mutuma, M., Rosand, B., Trindade, I., Bandaru, R., Cunningham, J., Xiong, C., Radev, D., & Radev, D. (2022). FeTaQA: Free-form Table Question Answering. Transactions of the Association for Computational Linguistics, 10, 35–49. doi:10.1162/tacl_a_00446.

Mueller, T., Piccinno, F., Shaw, P., Nicosia, M., & Altun, Y. (2019). Answering Conversational Questions on Structured Data without Logical Forms. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 5901–5909. doi:10.18653/v1/d19-1603.

Herzig, J., Nowak, P. K., Müller, T., Piccinno, F., & Eisenschlos, J. (2020). TaPas: Weakly Supervised Table Parsing via Pre-training. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. doi:10.18653/v1/2020.acl-main.398.

Zhu, F., Lei, W., Huang, Y., Wang, C., Zhang, S., Lv, J., Feng, F., & Chua, T.-S. (2021). TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. doi:10.18653/v1/2021.acl-long.254.

Sun, H., Ma, H., He, X., Yih, W., Su, Y., & Yan, X. (2016). Table Cell Search for Question Answering. Proceedings of the 25th International Conference on World Wide Web, 771 - 782. doi:10.1145/2872427.2883080.

Glass, M., Canim, M., Gliozzo, A., Chemmengath, S., Kumar, V., Chakravarti, R., Sil, A., Pan, F., Bharadwaj, S., & Fauceglia, N. R. (2021). Capturing Row and Column Semantics in Transformer Based Question Answering over Tables. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. doi:10.18653/v1/2021.naacl-main.96.

Chen, W., Chang, M. W., Schlinger, E., Wang, W., & Cohen, W. W. (2020). Open question answering over tables and text. arXiv, preprint arXiv:2010.10439.. doi:10.48550/arXiv.2010.10439.

Baik, C., Jagadish, H. V., & Li, Y. (2019). Bridging the Semantic Gap with SQL Query Logs in Natural Language Interfaces to Databases. 2019 IEEE 35th International Conference on Data Engineering (ICDE), abs 909 1765, 374–385. doi:10.1109/icde.2019.00041.

Xu, X., Liu, C., & Song, D. (2017). Sqlnet: Generating structured queries from natural language without reinforcement learning. arXiv preprint arXiv:1711.04436. doi:10.48550/arXiv.1711.04436.

Yu, T., Li, Z., Zhang, Z., Zhang, R., & Radev, D. (2018). Typesql: Knowledge-based type-aware neural text-to-SQL generation. arXiv preprint, arXiv:1804.09769. doi:10.48550/arXiv.1804.09769.

Iyer, S., Konstas, I., Cheung, A., Krishnamurthy, J., & Zettlemoyer, L. (2017). Learning a neural semantic parser from user feedback. arXiv preprint, arXiv:1704.08760. doi:10.48550/arXiv.1704.08760.

Gür, I., Yavuz, S., Su, Y., & Yan, X. (2018, July). Dialsql: Dialogue based structured query generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 1339-1349. doi:10.18653/v1/p18-1124.

Bogin, B., Berant, J., & Gardner, M. (2019). Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. doi:10.18653/v1/p19-1448.

Devlin, J. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint, arXiv:1810.04805. doi:10.48550/arXiv.1810.04805.

Yin, P., & Neubig, G. (2018). TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 7–12. doi:10.18653/v1/d18-2002.

Lin, K., Bogin, B., Neumann, M., Berant, J., & Gardner, M. (2019). Grammar-based neural text-to-sql generation. arXiv preprint, arXiv:1905.13326. doi:10.48550/arXiv.1905.13326.

Suhr, A., Chang, M.-W., Shaw, P., & Lee, K. (2020). Exploring Unexplored Generalization Challenges for Cross-Database Semantic Parsing. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. doi:10.18653/v1/2020.acl-main.742.

Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., ... & Zaremba, W. (2021). Evaluating large language models trained on code. doi:10.48550/arXiv.2107.03374.

Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., ... & Fedus, W. (2022). Emergent abilities of large language models. doi:10.48550/arXiv.2206.07682.

Popescu, A.-M., Etzioni, O., & Kautz, H. (2003). Towards a theory of natural language interfaces to databases. Proceedings of the 8th International Conference on Intelligent User Interfaces. doi:10.1145/604045.604070.

Min, Q., Shi, Y., & Zhang, Y. (2019). A Pilot Study for Chinese SQL Semantic Parsing. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 3650–3656. doi:10.18653/v1/d19-1377.

Tuan Nguyen, A., Dao, M. H., & Nguyen, D. Q. (2020). A Pilot Study of Text-to-SQL Semantic Parsing for Vietnamese. Findings of the Association for Computational Linguistics: EMNLP 2020. doi:10.18653/v1/2020.findings-emnlp.364.

Nguyen, D. Q., & Tuan Nguyen, A. (2020). PhoBERT: Pre-trained language models for Vietnamese. Findings of the Association for Computational Linguistics: EMNLP 2020. doi:10.18653/v1/2020.findings-emnlp.92.

Shi, P., Ng, P., Wang, Z., Zhu, H., Li, A. H., Wang, J., Nogueira dos Santos, C., & Xiang, B. (2021). Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15), 13806–13814. doi:10.1609/aaai.v35i15.17627.

Souza, F., Nogueira, R., Lotufo, R. (2020). BERTimbau: Pretrained BERT Models for Brazilian Portuguese. Intelligent Systems. BRACIS 2020. Lecture Notes in Computer Science, Springer, Cham, Switzerland. doi:10.1007/978-3-030-61377-8_28.

Tang, Y., Tran, C., Li, X., Chen, P.-J., Goyal, N., Chaudhary, V., Gu, J., & Fan, A. (2020). Multilingual Translation with Extensible Multilingual Pretraining and Finetuning. doi:10.48550/arXiv.2008.00401.

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023). Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. ACM Computing Surveys, 55(9), 1–35. doi:10.1145/3560815.

Turney, P. D. (2001). Thumbs up or thumbs down? Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL ’02, 417. doi:10.3115/1073083.1073153.

Vajjala, S., Majumder, B., Gupta, A., & Surana, H. (2020). Practical natural language processing: a comprehensive guide to building real-world NLP systems. O'Reilly Media, Sebastopol, United States.

Lafferty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. ICML, San Diego, United States.

Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine learning, 46, 389-422. doi:10.1023/A:1012487302797.

Och, F. J., Gildea, D., Khudanpur, S., Sarkar, A., Yamada, K., Fraser, A., ... & Radev, D. (2004). A smorgasbord of features for statistical machine translation. Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, 2-7 May, 2004, Boston, United States.

Zhang, Y., & Nivre, J. (2011). Transition-based dependency parsing with rich non-local features. Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, 19-24 June, 2011, Portland, United States.

Chollet, F. (2021). Deep learning with Python. Simon and Schuster, New York, United States.

Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Advances in neural information processing systems, MIT Press, Cambridge, United States.

Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014). A Convolutional Neural Network for Modelling Sentences. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. doi:10.3115/v1/p14-1062.

Kim, Y. (2014). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). doi:10.3115/v1/d14-1181.

Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving Language Understanding by Generative Pre-Training. Preprint. 1–12.

Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., ... & Hon, H. W. (2019). Unified language model pre-training for natural language understanding and generation. Advances in neural information processing systems, 8-14 December, Vancouver, Canada.

Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019). XLNet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems, MIT Press, Cambridge, United States.

Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020). BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. doi:10.18653/v1/2020.acl-main.703.

Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H. T., ... & Le, Q. (2022). Lamda: Language models for dialog applications. arXiv Preprint, arXiv:2201.08239.. doi:10.48550/arXiv.2201.08239.

Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., ... & Zettlemoyer, L. (2022). Opt: Open pre-trained transformer language models. arXiv Preprint. doi:10.48550/arXiv.2205.01068.

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., ... & Fiedel, N. (2023). Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240), 1-113.

Halevy, A., Norvig, P., & Pereira, F. (2009). The unreasonable effectiveness of data. IEEE Intelligent Systems, 24(2), 8–12. doi:10.1109/MIS.2009.36.

Wang, T., Roberts, A., Hesslow, D., Le Scao, T., Chung, H. W., Beltagy, I., ... & Raffel, C. (2022). What language model architecture and pretraining objective works best for zero-shot generalization?. International Conference on Machine Learning, PMLR, 7-23 July, 2022, Baltimore, United States.

Liao, Y., Jiang, X., & Liu, Q. (2020). Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 263–274. doi:10.18653/v1/2020.acl-main.24.

Zhao, Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021). Calibrate before use: Improving few-shot performance of language models. International Conference on Machine Learning. PMLR, 18-24 July, 2021, Virtual Event.

Holtzman, A., West, P., Shwartz, V., Choi, Y., & Zettlemoyer, L. (2021). Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 7038–7051. doi:10.18653/v1/2021.emnlp-main.564.

Arora, S., Narayan, A., Chen, M. F., Orr, L., Guha, N., Bhatia, K., ... & Ré, C. (2022). Ask me anything: A simple strategy for prompting language models. arXiv Preprint. doi:10.48550/arXiv.2210.02441.

Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022). Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 1-13. doi:10.18653/v1/2022.acl-long.556.

Reynolds, L., & McDonell, K. (2021). Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 1–7. doi:10.1145/3411763.3451760.

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, 28 November-9 December, 2022, New Orleans, United States.

Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., ... & Zhou, D. (2022). Self-consistency improves chain of thought reasoning in language models. arXiv Preprint. doi:10.48550/arXiv.2203.11171.

Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large language models are zero-shot reasoners. Advances in neural information processing systems. arXiv Preprint. doi:10.48550/arXiv.2205.11916.

Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N., & Lewis, M. (2023). Measuring and Narrowing the Compositionality Gap in Language Models. Findings of the Association for Computational Linguistics: EMNLP 2023. doi:10.18653/v1/2023.findings-emnlp.378.

Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., ... & Odena, A. (2021). Show your work: Scratchpads for intermediate computation with language models. arXiv Preprint. doi:10.48550/arXiv.2112.00114.

Jiang, Z., Xu, F. F., Araki, J., & Neubig, G. (2020). How can we know what language models know? Transactions of the Association for Computational Linguistics, 8, 423–438. doi:10.1162/tacl_a_00324.

Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019). Universal Adversarial Triggers for Attacking and Analyzing NLP. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 1-15. doi:10.18653/v1/d19-1221.

Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020). AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1-15. doi:10.18653/v1/2020.emnlp-main.346.

Gao, T., Fisch, A., & Chen, D. (2021). Making Pre-trained Language Models Better Few-shot Learners. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 1-15. doi:10.18653/v1/2021.acl-long.295.

Davison, J., Feldman, J., & Rush, A. (2019). Commonsense Knowledge Mining from Pretrained Models. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 1173-1178. doi:10.18653/v1/d19-1109.

Li, X. L., & Liang, P. (2021). Prefix-Tuning: Optimizing Continuous Prompts for Generation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 1-15. doi:10.18653/v1/2021.acl-long.353.

Zhong, Z., Friedman, D., & Chen, D. (2021). Factual probing is [mask]: Learning vs. learning to recall. arXiv Preprint, arXiv:2104.05240. doi:10.48550/arXiv.2104.05240.

Liu, X., Zheng, Y., Du, Z., Ding, M., Qian, Y., Yang, Z., & Tang, J. (2023). GPT understands, too. AI Open. doi:10.1016/j.aiopen.2023.08.012.

Gao L. (2021). On the Sizes of OpenAI API Models. EleutherAI, New York, United States. Available online: https://blog.eleuther.ai/gpt3-model-sizes/ (accessed on September 2024).

Ziegler, A. (2021). GitHub Copilot research recitation. The GitHub Blog, California, United States. Available online: https://github.blog/ai-and-ml/github-copilot/github-copilot-research-recitation/ (accessed on September 2024).


Full Text: PDF

DOI: 10.28991/ESJ-2024-08-05-020

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Marcelo Jannuzzi, Yuriy Perezhohin, Fernando Peres, Mauro Castelli