To what extent can AI be used in scientific writing?

Large language models (LLMs), the currently available AI tools for writing, are not yet sufficient for technical writing which required high accuracy and cited sources. There are many possible applications for AI and LLMs in scientific fields including writing, however, it does not possess logical reasoning capabilities.

For instance, consider this: when asked the question, "If it takes 3 towels hanging on a clothesline 3 hours to dry, how long will it take 9 towels to dry", GPT answers, "9 hours". It cannot contextualize information or draw logical conclusions - it is simply a text simulator. 

In summary, LLMs / AI have not yet progressed to the capability level for good scientific writing. Continue to explore AI capabilities with great caution.

To my mind AI is just another computer program and a tool for scientists. Its possible for anyone to use a scientific tool inappropriately and when we first started using computer programs instead of typewriters there was a HUGE problem with post graduate students "cutting and pasting"  from others' work. We solved it by writing programs that could identify plagiarism  and in the end they have been more successful and faster than those needed for written and typewritten documents. I edit a lot of post graduate theses and publications and have noted with much amusement how AI provides fictitious references if  given key words and asked to "search" . I am quite sure that AI can also be programmed  to efficiently detect plagiarism in future. HOWEVER  the current  AI  methodology needs  programmers who are scientists with scientific integrity; to make sure  that it is not used purposely or accidently to  sabotage scientific  honesty and rigidity.  Its like real life: we have intelligent criminals, so we need intelligent policing!!!

AI should be used with caution. It's fine to collect information and help non-english speakers. BUT the outcome is with numerous reviewers so everyone should be extremely careful. The errors are not just factual but also fake references a.o.

The AI may be used in research questions and hypothesis generation. It may be used in English editing to some extent but use of AI in content writing and excessive use of AI in writings will be a disaster. It will lead to dull minds and no one will be thinking to read or write or even extract information from literature. AI should be used as an assisting tool not leading to dependence.  

AI is helpful for drafting and rapid writing for non-native speakers. It is useful as a language tool. As for scientific content, it might be supportive. However, one has to re-check everything to avoid serious gaps and mistakes.

The use of AI should be limited to some research activities like literature searches, language editing, and plagiarism checks. However, AI nowadays can't be reliable to conduct and write research without human intervention.

AI is a good tool to refine your ideas. It best to write your thoughts and interpretation first. Then ask AI to refine the language. This strategy helps non-native English writers. 

We need to go in flow with the technology and should get familiarized with that and use it to the best for the benefit of mankind. As mentioned in other Pings here, the data should be verified thoroughly with an expert prior publishing. AI is a boon to mankind, heartfelt thanks to developers 

AI can be provide good assistance in scientific writing. A few places where it can be used during paper writing for conference/journal are:
1. For the literature review process, search recent research that are similar. 
2. Help rephrases part of a sentence that you feel sound amateurish/not suitable for scientific writing. You may also get suitable scientific terminologies with the help of AI.
3. Summarize the paper that will help writing abstract and conclusion sections.
Note that the AI should only assist in the above. Once you get the output from AI, rather than copy-pasting it, you need to rewrite it your own way 
One part where AI should not be used (at least before the first draft) in the beginning is in methodology/experiments/results section. These are the core of the paper and AI are known to suffer from hallucination. After completing the draft it can be used as explained in 2. and 3.

We can use current AI to develop broad structure that we edit ourselves.

In my country AI considers plagiarism so unfortunately, we cannot depend on AI in writing, but we can use it as a scientific tool. 

It seems to me it can help us in various level, but should be used mostly for language enhancement, data mining and analysis, and technical summarization.

if you use AI to write a paper, make a literature review for the introduction you should be aware that this will very likely be detected as plagiarism by journal checking systems. The best use I see is to simplify summarizing some information for the introduction of the discussion sections, that later can be modified by the authors. 
Of course, in the future AI will be more and more present in hypothesis generation and testing (see the debate about the Nobel-Turing Challenge). 

AI should be used in an extent that it does lead to plagiarism 

A.I. can be used as a simple model of our complicated brain. By enhancing its algorithms, it will become smarter. In a point, it will even perform better than our brain, and that time is when they can do everything containing writing like a human.

only for collecting data, check for plagiarism, sometimes to suggest hypothesis/research questions

Ai can be useful as long a human inputs information needed for it to generate rekevant data. Most of the time, we need to give the system datatnerate what we want. Human interaction will still be required all the time. 

I think AI should be able to help you rewrite paragraphs in manuscripts, acting like an editor and proofreader.  You do the initial work, and then AI performs the grammar checks, wordsmithing, and editing.  

AI can be used as an "aid", it can facilitate browsing of relevant subjects, it can help scientists organize data, etc. However, at this point, scientists still need to use their critical thinking and know what they are talking about. Currently, It is necessary to disclose the use of AI when writing the paper, what type of AI, what is does, etc. I discovered that, there are already softwares and AI services that identify if a researcher-writer used AI or not... so, whatever way one decides to uses AI for at this point (identifying novel and research gaps, methodology approaches, data interpretations, statistics, etc), AI itself is evolving in the areas of surveillance, ethical misconduct and plagiarism detection, novelty detection,  etc... So,  it is not advisable to blindly use it... We still need to do the work, use critical thinking, and follow ethical guidelines. 

It should be used as support to facilitate but for the moment you have to be careful when dealing with something that is not completely established or new contributions where dynamic analysis of the problem is important and where emotionality plays a role

Ai can be used in identifying and collecting related data but can not link data with each other. It can not generate novel thinking or hypothesis which a human brain can do.

AI is a very useful tool for academic writing but its use should not replace the human brain. If used, ethics demand that a disclosure should be acknowledged. It is also useful for better academic grammar, and content cohesion, and coherence. It can be used as a tool to assist but not to generate academic writing content. 

AI can be used chiefly for literature search data analysis and interpretation, in scientific writing.