{"id":14327,"date":"2023-07-05T00:00:30","date_gmt":"2023-07-05T00:00:30","guid":{"rendered":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/"},"modified":"2024-11-26T20:14:10","modified_gmt":"2024-11-26T20:14:10","slug":"building-a-conformal-chatbot-in-julia-1ed23363a280","status":"publish","type":"post","link":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/","title":{"rendered":"Building a Conformal Chatbot in Julia"},"content":{"rendered":"

Conformal Prediction, LLMs and HuggingFace – Part 1<\/h3>\n

Large Language Models (LLM) are all the buzz right now. They are used for a variety of tasks, including text classification, question answering, and text generation. In this tutorial, we will show how to conformalize a transformer language model for text classification using [ConformalPrediction.jl](https:\/\/juliatrustworthyai.github.io\/ConformalPrediction.jl\/dev\/)<\/code>.<\/p>\n

\ud83d\udc40 At a Glance<\/h2>\n

In particular, we are interested in the task of intent classification as illustrated in the sketch below. Firstly, we feed a customer query into an LLM to generate embeddings. Next, we train a classifier to match these embeddings to possible intents. Of course, for this supervised learning problem we need training data consisting of inputs – queries – and outputs – labels indicating the true intent. Finally, we apply Conformal Predition to quantify the predictive uncertainty of our classifier.<\/p>\n

Conformal Prediction (CP) is a rapidly emerging methodology for Predictive Uncertainty Quantification. If you’re unfamiliar with CP, you may want to first check out my 3-part introductory series on the topic starting with this post<\/a>.<\/p><\/blockquote>\n

<\/p>\n

\ud83e\udd17 HuggingFace<\/h2>\n

We will use the Banking77<\/a> dataset (Casanueva et al., 2020), which consists of 13,083 queries from 77 intents related to banking. On the model side, we will use the DistilRoBERTa<\/a><\/a> model, which is a distilled version of RoBERTa (Liu et al., 2019) fine-tuned on the Banking77 dataset.<\/p>\n

The model can be loaded from HF straight into our running Julia session using the [Transformers.jl](https:\/\/github.com\/chengchingwen\/Transformers.jl\/tree\/master)<\/code> package.<\/p>\n

This package makes working with HF models remarkably easy in Julia. Kudos to the devs! \ud83d\ude4f<\/em> <\/p><\/blockquote>\n

Below we load the tokenizer tkr<\/code> and the model mod<\/code>. The tokenizer is used to convert the text into a sequence of integers, which is then fed into the model. The model outputs a hidden state, which is then fed into a classifier to get the logits for each class. Finally, the logits are then passed through a softmax function to get the corresponding predicted probabilities. Below we run a few queries through the model to see how it performs.<\/p>\n

# Load model from HF \ud83e\udd17:\ntkr = hgf"mrm8488\/distilroberta-finetuned-banking77:tokenizer"\nmod = hgf"mrm8488\/distilroberta-finetuned-banking77:ForSequenceClassification"\n\n# Test model:\nquery = [\n    "What is the base of the exchange rates?",\n    "Why is my card not working?",\n    "My Apple Pay is not working, what should I do?",\n]\na = encode(tkr, query)\nb = mod.model(a)\nc = mod.cls(b.hidden_state)\nd = softmax(c.logit)\n[labels[i] for i in Flux.onecold(d)]<\/code><\/pre>\n
3-element Vector{String}:\n "exchange_rate"\n "card_not_working"\n "apple_pay_or_google_pay"<\/code><\/pre>\n
\ud83d\udd01  MLJ<\/code> Interface<\/h2>\n
Since our package is interfaced to [MLJ.jl](https:\/\/alan-turing-institute.github.io\/MLJ.jl\/dev\/)<\/code>, we need to define a wrapper model that conforms to the MLJ<\/code> interface. In order to add the model for general use, we would probably go through [MLJFlux.jl](https:\/\/github.com\/FluxML\/MLJFlux.jl)<\/code>, but for this tutorial, we will make our life easy and simply overload the MLJBase.fit<\/code> and MLJBase.predict<\/code> methods.<\/p>\n
Since the model from HF is already pre-trained and we are not interested in further fine-tuning, we will simply return the model object in the MLJBase.fit<\/code> method. The MLJBase.predict<\/code> method will then take the model object and the query and return the predicted probabilities. We also need to define the MLJBase.target_scitype<\/code> and MLJBase.predict_mode<\/code> methods. The former tells MLJ<\/code> what the output type of the model is, and the latter can be used to retrieve the label with the highest predicted probability.<\/p>\n
struct IntentClassifier &lt;: MLJBase.Probabilistic\n    tkr::TextEncoders.AbstractTransformerTextEncoder\n    mod::HuggingFace.HGFRobertaForSequenceClassification\nend\n\nfunction IntentClassifier(;\n    tkr::TextEncoders.AbstractTransformerTextEncoder, \n    mod::HuggingFace.HGFRobertaForSequenceClassification,\n)\n    IntentClassifier(tkr, mod)\nend\n\nfunction get_hidden_state(clf::IntentClassifier, query::Union{AbstractString, Vector{&lt;:AbstractString}})\n    token = encode(clf.tkr, query)\n    hidden_state = clf.mod.model(token).hidden_state\n    return hidden_state\nend\n\n# This doesn't actually retrain the model, but it retrieves the classifier object\nfunction MLJBase.fit(clf::IntentClassifier, verbosity, X, y)\n    cache=nothing\n    report=nothing\n    fitresult = (clf = clf.mod.cls, labels = levels(y))\n    return fitresult, cache, report\nend\n\nfunction MLJBase.predict(clf::IntentClassifier, fitresult, Xnew)\n    output = fitresult.clf(get_hidden_state(clf, Xnew))\n    p\u0302 = UnivariateFinite(fitresult.labels,softmax(output.logit)',pool=missing)\n    return p\u0302\nend\n\nMLJBase.target_scitype(clf::IntentClassifier) = AbstractVector{&lt;:Finite}\n\nMLJBase.predict_mode(clf::IntentClassifier, fitresult, Xnew) = mode.(MLJBase.predict(clf, fitresult, Xnew))<\/code><\/pre>\n
To test that everything is working as expected, we fit the model and generated predictions for a subset of the test data:<\/p>\n
clf = IntentClassifier(tkr, mod)\ntop_n = 10\nfitresult, _, _ = MLJBase.fit(clf, 1, nothing, y_test[1:top_n])\n@time y\u0302 = MLJBase.predict(clf, fitresult, queries_test[1:top_n]);<\/code><\/pre>\n
6.818024 seconds (11.29 M allocations: 799.165 MiB, 2.47% gc time, 91.04% compilation time)<\/code><\/pre>\n
Note that even though the LLM we’re using here isn’t really that large at all, even a simple forward pass does take considerable time.<\/p>\n
\ud83e\udd16 Conformal Chatbot<\/h2>\n
To turn the wrapped, pre-trained model into a conformal intent classifier, we can now rely on standard API calls. We first wrap our atomic model where we also specify the desired coverage rate and method. Since even simple forward passes are computationally expensive for our (small) LLM, we rely on Simple Inductive Conformal Classification.<\/p>\n
conf_model = conformal_model(clf; coverage=0.95, method=:simple_inductive, train_ratio=train_ratio)\nmach = machine(conf_model, queries, y)<\/code><\/pre>\n
Finally, we use our conformal LLM to build a simple yet powerful chatbot that runs directly in the Julia REPL. Without dwelling on the details too much, the conformal_chatbot<\/code> works as follows:<\/p>\n
    \n
  1. Prompt user to explain their intent.<\/li>\n
  2. Feed user input through conformal LLM and present the output to the user.<\/li>\n
  3. If the conformal prediction set includes more than one label, prompt the user to either refine their input or choose one of the options included in the set.<\/li>\n<\/ol>\n
    The following code implements these ideas:<\/p>\n
    function prediction_set(mach, query::String)\n    p\u0302 = MLJBase.predict(mach, query)[1]\n    probs = pdf.(p\u0302, collect(1:77))\n    in_set = findall(probs .!= 0)\n    labels_in_set = labels[in_set]\n    probs_in_set = probs[in_set]\n    _order = sortperm(-probs_in_set)\n    plt = UnicodePlots.barplot(labels_in_set[_order], probs_in_set[_order], title="Possible Intents")\n    return labels_in_set, plt\nend\n\nfunction conformal_chatbot()\n    println("\ud83d\udc4b   Hi, I'm a Julia, your conformal chatbot. I'm here to help you with your banking query. Ask me anything or type 'exit' to exit ...n")\n    completed = false\n    queries = ""\n    while !completed\n        query = readline()\n        queries = queries * "," * query\n        labels, plt = prediction_set(mach, queries)\n        if length(labels) &gt; 1\n            println("\ud83e\udd14 Hmmm ... I can think of several options here. If any of these applies, simply type the corresponding number (e.g. '1' for the first option). Otherwise, can you refine your question, please?n")\n            println(plt)\n        else\n            println("\ud83e\udd73 I think you mean $(labels[1]). Correct?")\n        end\n\n        # Exit:\n        if query == "exit"\n            println("\ud83d\udc4b   Bye!")\n            break\n        end\n        if query \u2208 string.(collect(1:77))\n            println("\ud83d\udc4d  Great! You've chosen '$(labels[parse(Int64, query)])'. I'm glad I could help you. Have a nice day!")\n            completed = true\n        end\n    end\nend<\/code><\/pre>\n
    Below we show the output for two example queries. The first one is very ambiguous (and misspelled as I just realised): "transfer mondey?". As expected, the size of the prediction set is therefore large.<\/p>\n
    ambiguous_query = "transfer mondey?"\nprediction_set(mach, ambiguous_query)[2]<\/code><\/pre>\n
                                                            Possible Intents              \n                                           \u250c                                        \u2510 \n                   beneficiary_not_allowed \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.150517   \n   balance_not_updated_after_bank_transfer \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.111409           \n                     transfer_into_account \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.0939535             \n        transfer_not_received_by_recipient \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.091163               \n            top_up_by_bank_transfer_charge \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.089306               \n                           failed_transfer \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.0888322              \n                           transfer_timing \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.0641952                   \n                      transfer_fee_charged \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.0361131                         \n                          pending_transfer \u2524\u25a0\u25a0\u25a0\u25a0\u25a0 0.0270795                           \n                           receiving_money \u2524\u25a0\u25a0\u25a0\u25a0\u25a0 0.0252126                           \n                         declined_transfer \u2524\u25a0\u25a0\u25a0 0.0164443                             \n                           cancel_transfer \u2524\u25a0\u25a0\u25a0 0.0150444                             \n                                           \u2514                                        \u2518 <\/code><\/pre>\n
    The following is a more refined version of the prompt: "I tried to transfer money to my friend, but it failed". It yields a smaller prediction set, since less ambiguous prompts result in lower predictive uncertainty.<\/p>\n
    refined_query = "I tried to transfer money to my friend, but it failed."\nprediction_set(mach, refined_query)[2]<\/code><\/pre>\n
                                                            Possible Intents              \n                                           \u250c                                        \u2510 \n                           failed_transfer \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.59042   \n                   beneficiary_not_allowed \u2524\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0\u25a0 0.139806                          \n        transfer_not_received_by_recipient \u2524\u25a0\u25a0 0.0449783                              \n   balance_not_updated_after_bank_transfer \u2524\u25a0\u25a0 0.037894                               \n                         declined_transfer \u2524\u25a0 0.0232856                               \n                     transfer_into_account \u2524\u25a0 0.0108771                               \n                           cancel_transfer \u2524 0.00876369                               \n                                           \u2514                                        \u2518 <\/code><\/pre>\n
    The video below shows the REPL-based chatbot in action. You can recreate this yourself and run the bot right from you terminal. To do so, just check out the original post<\/a> on my blog to find the full source code.<\/p>\n
    <\/p>\n
    \ud83c\udf2f  Wrapping Up<\/h2>\n
    This work was done in collaboration with colleagues at ING as part of the ING Analytics 2023 Experiment Week. Our team demonstrated that Conformal Prediction provides a powerful and principled alternative to top-K<\/em> intent classification. We won the first prize by popular vote.<\/p>\n
    Of course, there are a lot of things that can be improved here. As far as Large LMs are concerned, we have used a small one. In terms of Conformal Prediction, we have only looked at simple inductive conformal classification. This is a good starting point, but there are more advanced methods available, which are implemented in the package and were investigated during the competition. Another thing we did not take into consideration here is that we have many outcome classes and may in practice be interested in achieving class-conditional coverage. Stay tuned for more on this in future posts.<\/p>\n
    If you’re interested in finding out more about Conformal Prediction in Julia, go ahead and check out the repo<\/a> and docs<\/a>.<\/p>\n
    \ud83c\udf89  JuliaCon 2023 is around the corner and this year I will be giving a talk<\/a> about ConformalPrediction.jl.<\/em> Check out the details of my talk here<\/a> and have a look at the full jam-packed conference schedule<\/a>.<\/p><\/blockquote>\n
    \ud83c\udf93  References<\/h2>\n
    Casanueva, I\u00f1igo, Tadas Tem\u010dinas, Daniela Gerz, Matthew Henderson, and Ivan Vuli\u0107. 2020. "Efficient Intent Detection with Dual Sentence Encoders." In Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI<\/em> , 38\u201345. Online: Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/2020.nlp4convai-1.5<\/a><\/a>.<\/p>\n
    Liu, Yinhan, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. "RoBERTa: A Robustly Optimized BERT Pretraining Approach." arXiv. https:\/\/doi.org\/10.48550\/arXiv.1907.11692<\/a><\/a>.<\/p>\n
    \ud83d\udcbe  Data and Model<\/h3>\n
    The Banking77<\/a> dataset was retrieved from HuggingFace. It is published under the Creative Commons Attribution 4.0 International license (CC BY 4.0) and curated by PolyAI<\/a> and was originally published by Casanueva et al. (2020). With thanks also to Manuel Romero<\/a> who contributed the fine-tuned DistilRoBERTa<\/a> to HuggingFace.<\/p>\n
    \n
    Originally published at https:\/\/www.paltmeyer.com<\/a><\/a> on July 5, 2023.<\/em><\/p>","protected":false},"excerpt":{"rendered":"
    Conformal Prediction, LLMs and HuggingFace – Part 1 Large Language Models (LLM) are all the buzz right now. They are used for a variety of tasks, including text classification, question answering, and text generation. In this tutorial, we will show how to conformalize a transformer language model for text classification using [ConformalPrediction.jl](https:\/\/juliatrustworthyai.github.io\/ConformalPrediction.jl\/dev\/). \ud83d\udc40 At a […]<\/p>\n","protected":false},"author":18,"featured_media":14328,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"is_member_only":true,"sub_heading":"Conformal Prediction, LLMs and HuggingFace\u200a-\u200aPart 1","footnotes":""},"categories":[21,22],"tags":[3002,1303,942,450,446],"sponsor":[],"coauthors":[30610],"class_list":["post-14327","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-large-language-models","category-machine-learning","tag-conformal-prediction","tag-hugging-face","tag-julia","tag-large-language-models","tag-machine-learning"],"yoast_head":"\nBuilding a Conformal Chatbot in Julia | Towards Data Science<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Building a Conformal Chatbot in Julia | Towards Data Science\" \/>\n<meta property=\"og:description\" content=\"Conformal Prediction, LLMs and HuggingFace – Part 1 Large Language Models (LLM) are all the buzz right now. They are used for a variety of tasks, including text classification, question answering, and text generation. In this tutorial, we will show how to conformalize a transformer language model for text classification using [ConformalPrediction.jl](https:\/\/juliatrustworthyai.github.io\/ConformalPrediction.jl\/dev\/). \ud83d\udc40 At a […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/\" \/>\n<meta property=\"og:site_name\" content=\"Towards Data Science\" \/>\n<meta property=\"article:published_time\" content=\"2023-07-05T00:00:30+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-11-26T20:14:10+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg\" \/>\n\t<meta property=\"og:image:width\" content=\"2240\" \/>\n\t<meta property=\"og:image:height\" content=\"827\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Patrick Altmeyer\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@TDataScience\" \/>\n<meta name=\"twitter:site\" content=\"@TDataScience\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Patrick Altmeyer\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/\"},\"author\":{\"name\":\"TDS Editors\",\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/person\/f9925d336b6fe962b03ad8281d90b8ee\"},\"headline\":\"Building a Conformal Chatbot in Julia\",\"datePublished\":\"2023-07-05T00:00:30+00:00\",\"dateModified\":\"2024-11-26T20:14:10+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/\"},\"wordCount\":1093,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/towardsdatascience.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg\",\"keywords\":[\"Conformal Prediction\",\"Hugging Face\",\"Julia\",\"Large Language Models\",\"Machine Learning\"],\"articleSection\":[\"Large Language Models\",\"Machine Learning\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/\",\"url\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/\",\"name\":\"Building a Conformal Chatbot in Julia | Towards Data Science\",\"isPartOf\":{\"@id\":\"https:\/\/towardsdatascience.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg\",\"datePublished\":\"2023-07-05T00:00:30+00:00\",\"dateModified\":\"2024-11-26T20:14:10+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#primaryimage\",\"url\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg\",\"contentUrl\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg\",\"width\":2240,\"height\":827},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/towardsdatascience.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Building a Conformal Chatbot in Julia\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/towardsdatascience.com\/#website\",\"url\":\"https:\/\/towardsdatascience.com\/\",\"name\":\"Towards Data Science\",\"description\":\"Publish AI, ML & data-science insights to a global community of data professionals.\",\"publisher\":{\"@id\":\"https:\/\/towardsdatascience.com\/#organization\"},\"alternateName\":\"TDS\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/towardsdatascience.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/towardsdatascience.com\/#organization\",\"name\":\"Towards Data Science\",\"alternateName\":\"TDS\",\"url\":\"https:\/\/towardsdatascience.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/tds-logo.jpg\",\"contentUrl\":\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/tds-logo.jpg\",\"width\":696,\"height\":696,\"caption\":\"Towards Data Science\"},\"image\":{\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/TDataScience\",\"https:\/\/www.youtube.com\/c\/TowardsDataScience\",\"https:\/\/www.linkedin.com\/company\/towards-data-science\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/person\/f9925d336b6fe962b03ad8281d90b8ee\",\"name\":\"TDS Editors\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/towardsdatascience.com\/#\/schema\/person\/image\/23494c9101089ad44ae88ce9d2f56aac\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g\",\"caption\":\"TDS Editors\"},\"description\":\"Building a vibrant data science and machine learning community. Share your insights and projects with our global audience: bit.ly\/write-for-tds\",\"url\":\"https:\/\/towardsdatascience.com\/author\/towardsdatascience\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Building a Conformal Chatbot in Julia | Towards Data Science","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/","og_locale":"en_US","og_type":"article","og_title":"Building a Conformal Chatbot in Julia | Towards Data Science","og_description":"Conformal Prediction, LLMs and HuggingFace – Part 1 Large Language Models (LLM) are all the buzz right now. They are used for a variety of tasks, including text classification, question answering, and text generation. In this tutorial, we will show how to conformalize a transformer language model for text classification using [ConformalPrediction.jl](https:\/\/juliatrustworthyai.github.io\/ConformalPrediction.jl\/dev\/). \ud83d\udc40 At a […]","og_url":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/","og_site_name":"Towards Data Science","article_published_time":"2023-07-05T00:00:30+00:00","article_modified_time":"2024-11-26T20:14:10+00:00","og_image":[{"width":2240,"height":827,"url":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg","type":"image\/jpeg"}],"author":"Patrick Altmeyer","twitter_card":"summary_large_image","twitter_creator":"@TDataScience","twitter_site":"@TDataScience","twitter_misc":{"Written by":"Patrick Altmeyer","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#article","isPartOf":{"@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/"},"author":{"name":"TDS Editors","@id":"https:\/\/towardsdatascience.com\/#\/schema\/person\/f9925d336b6fe962b03ad8281d90b8ee"},"headline":"Building a Conformal Chatbot in Julia","datePublished":"2023-07-05T00:00:30+00:00","dateModified":"2024-11-26T20:14:10+00:00","mainEntityOfPage":{"@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/"},"wordCount":1093,"commentCount":0,"publisher":{"@id":"https:\/\/towardsdatascience.com\/#organization"},"image":{"@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#primaryimage"},"thumbnailUrl":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg","keywords":["Conformal Prediction","Hugging Face","Julia","Large Language Models","Machine Learning"],"articleSection":["Large Language Models","Machine Learning"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/","url":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/","name":"Building a Conformal Chatbot in Julia | Towards Data Science","isPartOf":{"@id":"https:\/\/towardsdatascience.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#primaryimage"},"image":{"@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#primaryimage"},"thumbnailUrl":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg","datePublished":"2023-07-05T00:00:30+00:00","dateModified":"2024-11-26T20:14:10+00:00","breadcrumb":{"@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#primaryimage","url":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg","contentUrl":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/07\/16sR4avVO7_a1P6V3Bsm6vg.jpeg","width":2240,"height":827},{"@type":"BreadcrumbList","@id":"https:\/\/towardsdatascience.com\/building-a-conformal-chatbot-in-julia-1ed23363a280\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/towardsdatascience.com\/"},{"@type":"ListItem","position":2,"name":"Building a Conformal Chatbot in Julia"}]},{"@type":"WebSite","@id":"https:\/\/towardsdatascience.com\/#website","url":"https:\/\/towardsdatascience.com\/","name":"Towards Data Science","description":"Publish AI, ML & data-science insights to a global community of data professionals.","publisher":{"@id":"https:\/\/towardsdatascience.com\/#organization"},"alternateName":"TDS","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/towardsdatascience.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/towardsdatascience.com\/#organization","name":"Towards Data Science","alternateName":"TDS","url":"https:\/\/towardsdatascience.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/towardsdatascience.com\/#\/schema\/logo\/image\/","url":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/tds-logo.jpg","contentUrl":"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2025\/02\/tds-logo.jpg","width":696,"height":696,"caption":"Towards Data Science"},"image":{"@id":"https:\/\/towardsdatascience.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/TDataScience","https:\/\/www.youtube.com\/c\/TowardsDataScience","https:\/\/www.linkedin.com\/company\/towards-data-science\/"]},{"@type":"Person","@id":"https:\/\/towardsdatascience.com\/#\/schema\/person\/f9925d336b6fe962b03ad8281d90b8ee","name":"TDS Editors","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/towardsdatascience.com\/#\/schema\/person\/image\/23494c9101089ad44ae88ce9d2f56aac","url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","caption":"TDS Editors"},"description":"Building a vibrant data science and machine learning community. Share your insights and projects with our global audience: bit.ly\/write-for-tds","url":"https:\/\/towardsdatascience.com\/author\/towardsdatascience\/"}]}},"distributor_meta":false,"distributor_terms":false,"distributor_media":false,"distributor_original_site_name":"Towards Data Science","distributor_original_site_url":"https:\/\/towardsdatascience.com","push-errors":false,"_links":{"self":[{"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/posts\/14327","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/comments?post=14327"}],"version-history":[{"count":0,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/posts\/14327\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/media\/14328"}],"wp:attachment":[{"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/media?parent=14327"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/categories?post=14327"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/tags?post=14327"},{"taxonomy":"sponsor","embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/sponsor?post=14327"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/towardsdatascience.com\/wp-json\/wp\/v2\/coauthors?post=14327"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}