• 0 Posts
  • 39 Comments
Joined 1 year ago
cake
Cake day: June 14th, 2023

help-circle
  • Now instead of just querying the goddamn database, a one line fucking SQL statement, I have to deal with the user team

    Exactly, you understand very well the purpose of microservices. You can submit a patch if you need that feature now.

    Funnily enough I’m the technical lead of the team that handles the user service in an insurance company.

    Due to direct access to our data without consulting us, we’re getting legal issues as people were using addresses to guess where people lived instead of using our endpoints.

    I guess some people really hate the validation that service layers have.


  • I’m afraid that would not be sufficient.

    These instructions are a small part of what makes a model answer like it does. Much more important is the training data. If you want to make a racist model, training it on racist text is sufficient.

    Great care is put in the training data of these models by AI companies, to ensure that their biases are socially acceptable. If you train an LLM on the internet without care, a user will easily be able to prompt them into saying racist text.

    Gab is forced to use this prompt because they’re unable to train a model, but as other comments show it’s pretty weak way to force a bias.

    The ideal solution for transparency would be public sharing of the training data.




  • It’s absolutely amazing, but it is also literally and technologically impossible for that to spontaneously coelesce into reason/logic/sentience.

    This is not true. If you train these models on game of Othello, they’ll keep a state of the world internally and use that to predict the next move played (1). To execute addition and multiplication they are executing an algorithm on which they were not explicitly trained (although the gpt family is surprisingly bad at it, due to a badly designed tokenizer).

    These models are still pretty bad at most reasoning tasks. But training on predicting the next word is a perfectly valid strategy, after all the best way to predict what comes after the “=” in 1432 + 212 = is to do the addition.









  • Yes to your question, but that’s not what I was saying.

    Here is one of the most popular training datasets : https://pile.eleuther.ai/

    If you look at the pdf describing the dataset, you’ll find the mean length of these documents to be somewhat short with mean length being less than 20kb (20000 characters) for most documents.

    You are asking for a model to retain a memory for the whole duration of a discussion, which can be very long. If I chat for one hour I’ll type approximately 8400 words, or around 42KB. Longer than most documents in the training set. If I chat for 20 hours, It’ll be longer than almost all the documents in the training set. The model needs to learn how to extract information from a long context and it can’t do that well if the documents on which it trained are short.

    You are also right that during training the text is cut off. A value I often see is 2k to 8k tokens. This is arbitrary, some models are trained with a cut off of 200k tokens. You can use models on context lengths longer than that what they were trained on (with some caveats) but performance falls of badly.


  • There are two issues with large prompts. One is linked to the current language technology, were the computation time and memory usage scale badly with prompt size. This is being solved by projects such as RWKV or mamba, but these remain unproven at large sizes (more than 100 billion parameters). Somebody will have to spend some millions to train one.

    The other issue will probably be harder to solve. There is less high quality long context training data. Most datasets were created for small context models.




  • As long as the demographic chart of Palestinians murdered by the IDF looks like the actual Palestinian population demographic (1/3 women, 1/3 kids) it’s safe to assume that there is absolutely no real targeting taking place.

    Yes, there is a bump if you look at the Hamas fighting population demographics but it is a minority. The large majority of people killed in this war are civilians there is no doubt about that. I was denying the 1:100 figure. For example Hamas has 1\3 of female victims, yet have a 1:4 casualty rate.

    Netanyahu literally said publicly that he saw wants to kill all Palestinians including the women and children and his deeds match his words.

    No he didn’t and you know it. Why lie ?

    Some senior Hamas executives have had such a discourse for Jews before being very softly reprimanded by Hamas but no executive from the Israeli government. There have been plenty of dog whistles, but they are not stupid enough to say it literally.

    Edit : I didn’t realize it but you were the person calling for the massacre of civilians in an earlier comment. Explains why you would lie, you need to dehumanise your enemy. I’m not spending more energy on this. You’re too far gone.



  • Where did I say that one side didn’t want to genocide the other ? Hamas is more public about it and won’t even try to justify their civilian killings, but Netanhyu government has made it clear again and again that they are willing to do collective punishment. The high civilian death rate is of course intentional.

    Hamas has also killed plenty of civilians, and they don’t even try to pretend that it was accidental. That said you are close to their ratio which is three civilians for every military death.

    Israel’s civilian deaths to militant deaths is probably higher due to the usage of bombs (10 civilian deaths per explosion) and intentional starvation but it isn’t 100:1.

    Hamas’ strategy of hiding behind civilians is also a war crime since it obviously increases the number of civilians killed.

    If you believe Israeli propaganda, they have killed 5000 Hamas militants. Reality is probably smaller than that, but since Hamas intentionally doesn’t publish their militant casualties we won’t have a good estimation. That said 500 Israeli soldiers have died and seeing the asymmetry in warfare, you can expect much more Hamas militants to have died. I have not been able to find an estimate from an independent source.