Ηow to study effectively with ChatGPT

Alex Avdiushenko

aleksandr.avdiushenko@jetbrains.com

1. State-of-the-art in solving math problems

2. Student's experience with AI

3. Teacher's experience

Race for AGI

Source: The State of AI Report

GPT-4 by OpenAI: one year ago
Gemini by Google AI: two months ago
Claude 3 by Anthropic: two months ago

And also LLaMa, Mistral, Grok…

Comparison of AI Models

Feature	GPT-4	Gemini	Claude 3
Model Size	1.76 trillion parameters	100 trillion	Unknown
Key Features	Advanced language understanding	Customizable for specific tasks, 1M+ context	Human-like interaction
Applications	Content creation, coding assistance	Data analysis, predictive tasks	Customer support, virtual assistance

Let's break down the key aspects of each three models to see how they stack up against each other and what that means for users like us in the academic and technological fields. In terms of model size, GPT-4 has an impressive 1.76 trillion parameters, making it one of the most powerful models in terms of raw processing and complexity. Gemini, on the other hand, offers variable sizes up to 100 trillion, allowing customization based on the user’s specific needs. Meanwhile, the specifics of Claude 3’s model size remain undisclosed, hinting at a potentially different approach or proprietary technology. Focusing on their key features, GPT-4 excels in advanced language understanding, which is crucial for generating human-like text and complex problem-solving. Gemini is designed to be highly customizable, suitable for specific tasks that may require a tailored approach. Claude 3 is famous for its human-like interactions, making it ideal for applications involving conversational AI. Regarding applications, GPT-4 has been great in content creation and coding assistance. Gemini shines in data analysis and predictive tasks, providing robust tools for researchers and data scientists. Claude 3 finds its strength in customer support and serving as a virtual assistant.

Results are impressive

AlphaGeometry, 17 Jan 2024: An Olympiad-level AI system for school tasks geometry

Solves 25 out of 30 latest IMO geometry problems
The previous best method only solves 10 problems
By using existing symbolic engines on a diverse set of random theorem premises, they extracted 100 million synthetic theorems and their proofs, many with more than 200 proof steps, four times longer than the average proof length of olympiad theorems
Then produced nearly 10 million synthetic proof steps that construct auxiliary points, reaching beyond the scope of pure symbolic deduction

And the state-of-the-art results are truly impressive. Let's start with a groundbreaking paper published in January 2024, in 'Nature'. This study introduces us to AlphaGeometry, an olympiad-level AI system designed specifically to tackle complex school geometry tasks.

AlphaGeometry has achieved a remarkable result by successfully solving 25 out of the 30 geometry problems from the International Mathematical Olympiad.

To put this achievement into context, the previous best method could only solve 10 problems, and average human doesn't solve even one problem.

But how did AlphaGeometry reach such a proficiency? By harnessing existing symbolic engines and applying them to a diverse set of random theorem premises, the AI managed to extract an astonishing 100 million synthetic theorems and their corresponding proofs. So here researchers managed to add to the AI system a deep understanding of school geometry.

Moreover, AlphaGeometry produced nearly 10 million synthetic proof steps that involve constructing auxiliary points. The AI is not just following a set of rules; it's exhibiting an understanding of geometry that allows it to reach solutions in a way that's reminiscent of human intuition and creativity.
The implications of this advancement are profound. They not only revolutionize the way we approach mathematical problems but also pave the way for future applications of AI in education and beyond.

In this slide, you see two parts: On the left, we begin with a simple problem from high school geometry, where triangle ABC has two equal sides, AB and AC, and we are required to prove that the angles opposite these sides are also equal. It's a basic problem that tests understanding of isosceles triangles. The process starts with the AI constructing the necessary points and lines, as shown in the middle section of the diagram, where it strategically decides on constructions and deductions needed to reach a solution. When the AI encounters a dead-end, it retraces its steps and tries different approaches until it finds a viable solution path. Moving to a more complex scenario, the image below outlines the AI’s approach to solving a problem from the 2015 International Mathematical Olympiad. This problem involves constructing points, understanding properties of circles and tangents, and proving that two circumcircles are tangent to each other. In the last Solution section, we see the key construction steps, which are crucial for solving the problem for both human and machine. So AlphaGeometry invites us to reimagine the future of education and academic competition, making it an exciting time for both mathematics, teachers and students.

Even now general ChatGPT is quite good in olympiad math

It turns out that now you have a brilliant AI assistant in your pocket!

it helps guide your thinking
it can search for relevant information
it even generates insights

You can focus only on the critical thinking part.

For example, first prompts

You are an expert at C++. How would you recommend a beginner to learn C++? Create a syllabus and lesson plan for me to learn this information effectively.

Also give me a list of bullets with other topics and concepts I should explore to understand C++ more completely and also tell me why.

1. State-of-the-art in solving math problems

2. Student's experience with AI

3. Teacher's experience

Student's cases of using AI assistance

One of the exams had the format “there is a realistic case study, you need to describe the actions necessary to solve it.” We had only two samples from the teacher: 2 situations and 2 expected answers to them. So we added the theory from the slides of all the lectures into the GPT, followed it with exam samples and asked it to generate similar tasks and answers to them based on the received theory. Super fast preparation for this kind of exam))

The process of completing practical homework with writing code has now changed: ChatGPT writes basic, often broken, code, which I take as the basis for further execution of the task.
In the further process of execution, Yaroslav uses it as a bug fixer — it is not always easy to understand what the error is. So you can throw the code into it and ask to find and fix the error.

And of course, students master new technologies just as well as teachers or even better.
Consider two typical examples from students of our educational program in Cyprus. NADEZHDA with her friends had one exam in the format of solving practical cases. They had to prepare using only two task samples from the teacher. So they added the theory from the slides of all the lectures into the GPT, followed it with exam samples. And asked AI to generate similar task-samples based on the received theory. It was a superfast preparation for this kind of exam!
Let's look at another example from YAROSLAV. He was given some task to write some kind of convolutional neural network. Initially, when looking at such a task, it is not always clear how to approach it and where to start. In the past, Google solved this, helping with existing solutions and showing how related tasks are done.
Now ChatGPT just writes the first version of code. Usually, approximately 20% of this code remains in the final version of neural network; and sometimes it even writes completely inadequate code, which, however, helps to understand what to Google next. In the following process of execution, Yaroslav uses ChatGPT as a bug fixer.

Pros and cons of ChatGPT in studying

Useful cases:

helps answer questions for tools with poor or unreadable documentation (for example, it can tell you how to start using GDB)

gives good and quick examples for some JSONs, XMLs with the necessary characteristics, or some fixtures to fill in the database

helps to overcome the fear of a “blank slate”

What it doesn't handle well:

any task where you need to understand deeply the general context of the whole project

not typical problems in not so popular languages: if you ask to solve a dynamic programming problem in Haskell, it gives funny wrong answers

for different mathematical facts, a lot of the GPT-generated proofs are incorrect

So what are the pros and cons of using ChatGPT in studying?
Let's begin with the USEFUL CASES.
It HELPS answer questions for tools with poor or inaccessible documentation (for example, it can tell you how to start using GNU Debugger).
It also PROVIDES nice and quick examples for some JSONs and other objects, or some values to fill in the database.
ONE of the main points: it helps to overcome the fear of a “blank slate.”
And WHAT it doesn't handle well:
Definitely it is ANY TASK where you have a large general context of the whole project.
ALSO, it is non-typical problems in not so popular areas: if you ask to solve a dynamic programming problem in Haskell, it gives funny answers, which are totally wrong.
And CRUCIAL for us: for different mathematical facts a lot of the GPT-generated proofs are still incorrect.

Will AI replace me?

nsk_illustration

2003, silver medal pms

2009, master of math mmf

2014, Ph.D. ict

2016, YDS shad

2015-2018, analyst
yandex

2016-2022, ed manager
csc

2023-..., lecturer & researcher

GPT-3.5

GPT-4v

? GPT-4o ?

pay_the_bills

Let's switch gears and dive into my personal story and also let's try to think, if or when AI replace me?
I was BORN and raised in faraway snowy Novosibirsk in a beautiful Soviet family: mother is a doctor, father is an engineer, also I have an older brother.
Childhood in the 90s was fun: unfinished buildings as playground, the sea and the forest instilled a love of sports and tourism. In school, about the sixth grade I tried mathematics, already in the seventh grade I won the regional math olympiad. It was almost a miracle. Away we go — a physics and mathematics school in the legendary AKADEMGORODOK. Then there was MATH at Novosibirsk State University, science... And the MOTTO was "You could not be a scientist, but a candidate of science must be!” =) So I made my PhD thesis in math modeling in the Institute of Computational Techonlogies.
In graduate school, I wanted to try something else. In parallel, the hype about machine learning grew; a YANDEX company wanted to come into Novosibirsk with the development office. This chance could not be missed, and I was fortunate to simultaneously become a head and a student of the Yandex School Novosibirsk branch. It was truly challenging: I even took academic leave for the first time in my life. But I liked everything and was happy :) I already worked as a data scientist at YANDEX at the graduation from the Data School.
It was beneficial for about three years of experience in industrial programming and analytics, but I became increasingly aware that I was drawn back to teaching, science and organizing education.
So I moved to Saint Petersburg and started working as a HEAD of the machine learning bachelor's program in the new department of mathematics and computer science of St. Petersburg State University. It was such an educational startup within the university — a very strong team, support from different tech companies as JetBrains, Gazprom Neft and other companies, many ambitious tasks and one big goal to create something like MIT in Russia.
Now I am working here in Cyprus at JetBrains, developing a new bachelor's program "Computer Science and Artificial Intelligence" at Neapolis University Paphos.
So now goes an intriguing question, can or when does AI replace me?
The researchers tested GPT-4v, the previous best model from OpenAI before GPT-4o, on reasoning problems found in intelligence tests and exams like the SAT. Notably, SAT is a standardized test used for college admissions in the United States. It is a multiple-choice, pencil-and-paper test that measures a student's reading, writing, and math skills. Surprisingly, GPT-4v performed impressively well, getting 700 out of 800 points on the math SAT. This outperformed the average score of human participants. Personally, I tested GPT-4v on some assignments for university students, and it solved a lot of them successfully. So, it seems that AI is already capable of replacing me in some tasks, and we don't know what will be in the future.
Anyway, I can still DRIVE a car, somehow understand and talk to people, and the only hope that it will help me pay the bills.

1. State-of-the-art in solving math problems

2. Student's experience with AI

3. Teacher's experience

Demo: pdf-files creation in 2K24

PyCharm with TeXiFy IDEA plugin — LaTeX, Python and JavaScript in one place with great autocompletion, inverse search from PDF, git etc.

AI assistant helps with translations in a broad sense: not only between languages, but also any formats, short ideas to nice descriptions, text to speech, auto tests creation, etc.

GitHub Copilot plugin offers very reasonable pieces of code, draft solutions, and generally helps speed up writing text content

Grazie plugin corrects typos and polishes long sentences

Now let me share my own workflow for teaching and lecture preparation.
FIRSTLY, I usually work in PyCharm complemented with the TeXiFy IDEA plugin. This combination brings together LaTeX, Python, and JavaScript all in one place. It allows using powerful autocompletion and inspections of PyCharm, and inverse search from pdf.
NEXT, let's discuss the AI Assistant. It is capable of assisting with translations in a broader sense. AI helper is not just restricted to translating languages; it can convert formats, elaborate short ideas into detailed descriptions, and even to oral speech, facilitate auto tests creation, and more.
THIRDLY, I'm using the GitHub Copilot plugin. It offers practical pieces of code, draft solutions, and generally helps speed up the writing of text content for my lectures and other materials.
LAST but not least, the Grazie plugin checks for typos and polishes long sentences. Its purpose is to ensure your text is clean, concise, and clear before reaching your audience.

Here I'd like to show the concrete example from my own experience. It is producing the problem for the International Mathematical Competition.
First of all, I have chosen an appropriate problem from ancient times of my teaching at Novosibirsk State University. There I organized math contests with tricky olimpiad tasks. So I need to translate the problem from Russian to English and to make a Latex format from Microsoft Word.
You see, with the help of AI assistant; I can do it in a few seconds. Here I simply copy-paste the answer to the working area. Then I check the result and look at the Grazie suggestions. Here I don't accept it, because it suggests something strange about additional dot in the text after the word "elements", but often it helps to polish the text.
Then I compile Tex sources and look at PDF file.
Then I need to write a solution, so I think through the solution and type it in Latex format. So firstly, I write x^2 = 0, then latex Rightarrow command goes. Here you can see that GitHub Copilot helps me to write the solution very quickly; it even makes simple calculation with math formulas.
Then I write the second part of the solution. Here I need to prove that the product being equal to zero is commutative. So that ab=0 if and only if ba=0. You see, sometimes I disagree with the Copilot's suggestions, but often it helps. Eventually, I check everything, recompile Tex sources and get the final PDF file done. Really fast, effective and convenient workflow, at least for me. Maybe you will consider using something similar for your own practice.

Few observations from the Academy about studying

Incredible speeds: any new knowledge is immediately available to everybody

Yesterday we found out smth on Twitter $\to$ tomorrow added a new skill to ourselves $\to$ in a week all the students have it

Therefore solving unsolved problems is the key skill

Essential activities for critical thinking development are

Mathematical proofs

Debugging

System engineering

Human discussions

Let's go on to a few observations about modern education from our teachers.

They NOTED the incredible speed of learning today. Essentially, as soon as new knowledge emerges, it becomes almost immediately available to everyone. This speed makes a significant impact.
For instance, IMAGINE we discover something on Twitter today. By tomorrow, we've already added a new skill to our repertoire. In just a week, all of our students have acquired this skill.
That's why the CRUCIAL skill nowadays becomes the ability to solve unsolved problems.
ALSO our teachers highlighted certain essential activities for the development of critical thinking.
Those activities include crafting mathematical PROOFS and checking them,
DEBUGGING problems in code or systems,
tasks related to system ENGINEERING, including robotics,
and, importantly, you have to engage in HUMAN discussions and debates.
Each activity, in its own way, sharpens our ability to evaluate information critically and make reasoned judgments.

Another four AI-powered cases from teachers

Creating lecture outline (click on the link!) $\to$ Making slides in reveal.js

Checking how easily an assignment task can be answered with the help of LLM

DALL-E 3 draws beautiful high-quality illustrations, in raster or in vector formats

Sending all routine tasks in a broad sense to the ChatGPT (e.g., writing syllabus draft)

Here in the final part of the talk, let's look at the practical use-cases of AI that teachers are utilizing right now.
FIRSTLY, AI can significantly streamline the creation of lecture materials. With a simple prompt, AI can help you generate a detailed lecture outline and even convert that outline into visually engaging slides using tools like reveal.js.
NEXT, teachers are experimenting with AI's capabilities by checking the effectiveness of LLM in answering assignment questions. This not only tests the AI's understanding but also helps to identify how students might use such technology to aid their learning.
In the realm of ART and visuals, DALL-E 3 is another remarkable tool. It's capable of creating nice, high-quality illustrations, whether in raster or in vector formats and even change it according to your opinion. These illustrations can then be used to enhance teaching materials and engage students.
LASTLY, AI is taking on the task of handling routine responsibilities. From drafting course syllables to managing other repetitive tasks, AI is opening up space for teachers to focus on more personalized and creative aspects of their work.

Instead of a conclusion

Reliable information sources have evolved, but the need for critical thinking and verification remains constant or even more important

Incredible speeds give us incredible opportunities

JetBrains has an amazing IDEs for everyone and AI, and this is cool!

We strive to pioneer in integrating AI in education tools also, shaping a smarter, more efficient learning environment

Come and join us — it's fun and gives great development

🎉🥳🎉 Text me anything! 🎉🥳🎉

Now, as we reach the final of the presentation, I'd like to leave you with some takeaways.
FIRSTLY, the panorama of reliable information sources has evolved over the centuries. Yet one constant remains — the need for critical thinking and verification. In today's fast-paced world, this need is not just constant but has become more crucial than ever.
SECONDLY, the incredible speed at which information is now accessible presents us with new opportunities. It allows for rapid learning, immediate access to knowledge, and the ability to connect globally. However, it also demands from us a strong sense of discernment and responsibility.
As a COMPANY, addressing these changes means adapting and innovating. We already have 'JetBrains AI assistant', integrated across all the products, improving our workflows and decision-making processes.
FINALLY, we don't just adapt but lead. We have the opportunity to be pioneers in deep integration of AI into software development and education tools, both online and offline. By doing this, we can shape a smarter, more efficient working and learning environment.
So, I invite you to JOIN us in studying and in development new learning environment; it's really fun and exciting.
Your thoughts, FEEDBACK, and ideas are invaluable. I encourage you to connect with me on Telegram or GitHub. Let's collaborate and shape the future of education together. Thank you very much for you attention.