Do you think there’s any reason to believe that these tools are going to continue their breakneck progress? It seems like we’ve reached a point where throwing more GPUs and text at these things is not yielding more results, and they still don’t have the problem solving skills to work out tasks outside of their training set. It’s closer to a StackOverflow that magically has the answers to most questions you ask than a replacement for proper software engineering. I know you never know if a breakthrough is around the corner, but it feels like we’ve hit a plateau for the foreseeable future.

It’s very good for navigating and editing text quickly, and fantastic for situations like “I need to do the same thing 100 times” with things like macros. Coders are frequently opening a big, complex file, jumping around it a lot, changing big and small parts of it, and doing repetitive tasks. For something more like writing out thoughts for an email, editing them slightly, then being done with that text forever, there aren’t as many advantages, you’re spending most of your time in “insert” mode which is effectively “normal text editor that people are used to” mode. That said, it’s one of those things where when you do get used to it and start to enjoy it instead of being frustrated by how different it is, you start wanting it wherever you have to type anything.

They also took away the ability to specify your answer separately from the answer you were looking for from others, so now it’s just “did you say the same thing.” Which doesn’t make any sense for some questions, like “do you prefer a partner that is a) taller than you, b) shorter than you, c) doesn’t matter”, if you both picked A or B, you aren’t a match for this question!

I think you’re thinking of SerenityOS (although it isn’t actually a Linux):

https://arstechnica.com/gadgets/2021/08/not-a-linux-distro-review-serenityos-is-a-unix-y-love-letter-to-the-90s/

Yeah, they’re probably talking about nulls. In Java, object references (simplified pointers, really) can be null, pointing nowhere and throwing an exception if you try to access them, which is fine when you don’t have a value for that reference (for example, you asked for a thing that doesn’t exist, or you haven’t made the thing yet), but it means that every time you interact with an object, if it turns out to have been null, a null pointer exception is getting thrown and likely crashing your program. You can check first if you think a value might be null, but if you miss one, it explodes.

Kotlin has nulls too, but the type system helps track where they could be. If a variable can be null, it’ll have a type like String?, and if not, the type is String. With that distinction, a function can explicitly say “I need a non-null value here” and if your value could be null, the type system will make you check first before you can use it.

Kotlin also has some nice quality of life improvements over Java; it’s less verbose (not a hard task), doesn’t force everything to belong to a class, supports data classes which are automatically immutable and behave more like primitive values than objects, and other improvements.

I’m bitterly clinging to my iPhone 13 mini, because I suspect it’s the last phone I’ll ever actively enjoy. I went along with bigger phones when that became the trend and decided I didn’t like them, and the mini line was such a relief to go back to. Once it’s no longer tenable, I’ll probably just buy a series of “the least bad used phone I can find” because I know I’ll be mildly frustrated every time I use it.

I’m still using an iPhone mini and I haven’t experienced any bad layouts, broken websites, or any difficulty like that. It has the same resolution of the biggest iPhone I’ve ever had (iPhone X) so things are smaller, which would make it a poor fit for someone with poor vision, but for me it’s an absolutely perfect phone. It’s frustrating to know that the perfect phone for me could easily exist, and yet Apple will refuse to make it for me. I’ll be stuck with phones I don’t like for the rest of my life, it seems.

I see this as an accessibility problem, computers have incredible power but taking advantage of it requires a very specific way of thinking and the drive to push through adversity (the computer constantly and correctly telling you “you’re doing it wrong”) that a lot of people can’t or don’t want to do. I don’t think they’re wrong or lazy to feel that way, and it’s a barrier to entry just like a set of stairs is to a wheelchair user.

The question is what to do about it, and there’s so much we as an industry should be doing before we even start to think about getting “normies” writing code or automating their phones. Using a computer sucks ass in so many ways for regular people, you buy something cheap and it’s slow as hell, it’s crapped up with adware and spyware out of the box, scammers are everywhere ready to cheat you out of your money… anyone here is likely immune to all that or knows how to navigate it but most people are just muddling by.

If we got past all that, I think it’d be a question of meeting users where they are. I have a car but I couldn’t replace the brakes, nor do I want to learn or try to learn, but that’s okay. My car is as accessible as I want it to be, and the parts that aren’t accessible, I go another route (bring it to a mechanic who can do the things I can’t). We can do this with computers too, make things easy for regular people but don’t try to make them all master programmers or tell them they aren’t “really” using it unless they’re coding. Bring the barrier down as low is it can go but don’t expect everyone to be trying to jump over it all the time, because they likely care about other things more.

I’m so confused that the same people can say “why does everyone get their undies in a bunch that we happily accept putting arbitrary data in columns regardless of type, that’s good, it’s flexible, but fine, we’ll put in a ‘strict’ keyword if you really want column types to mean something” and also “every other SQL says 1==‘1’ but this is madness, strings aren’t integers, what is everyone else thinking?!”

“Life without risk of poverty”?! That desperation and fear is the only way I can staff my sweatshops!

Back in the olden days, if you wrote a program, you were punching machine codes into a punch card and they were being fed into the computer and sent directly to the CPU. The machine was effectively yours while your program ran, then you (or more likely, someone who worked for your company or university) noted your final results, things would be reset, and the next stack of cards would go in.

Once computers got fast enough, though, it was possible to have a program replace the computer operator, an “operating system”, and it could even interleave execution of programs to basically run more than one at the same time. However, now the programs had to share resources, they couldn’t just have the whole computer to themselves. The OS helped manage that, a program now had to ask for memory and the OS would track what was free and what was in use, as well as interleaving programs to take turns running on the CPU. But if a program messed up and wrote to memory that didn’t belong to it, it could screw up someone else’s execution and bring the whole thing crashing down. And in some systems, programs were given a turn to run and then were supposed to return control to the OS after a bit, but it was basically an honor system, and the problem with that is likely clear.

Hardware and OS software added features to enforce more order. OSes got more power, and help from the hardware to wield it. Now instead of asking politely to give back control, the hardware would enforce limits, forcing control back to the OS periodically. And when it came to memory, the OS no longer handed out addresses matching the RAM for the program to use directly, instead it could hand out virtual addresses, with the OS tracking every relationship between the virtual address and the real location of the data, and the hardware providing Memory Management Units that can do things like store tables and do the translation from virtual to physical on its own, and return control to the OS if it doesn’t know.

This allows things like swapping, where a part of memory that isn’t being used can be taken out of RAM and written to disk instead. If the program tries to read an address that was swapped out, the hardware catches that it’s a virtual address that it doesn’t have a mapping for, wrenches control from the program, and instead runs the code that the OS registered for handling memory. The OS can see that this address has been swapped out, swap it back in to real RAM, tell the hardware where it now is, and then control returns to the program. The program’s none the wiser that its data wasn’t there a moment ago, and it all works. If a program messes up and tries to write to an address it doesn’t have, it doesn’t go through because there’s no mapping to a physical address, and the OS can instead tell the program “you have done very bad and unless you were prepared for this, you should probably end yourself” without any harm to others.

Memory is handed out to programs in chunks called “pages”, and the hardware has support for certain page size(s). How big they should be is a matter of tradeoffs; since pages are indivisible, pages that are too big will result in a lot of wasted space (if a program needs 1025 bytes on a 1024-byte page size system, it’ll need 2 pages even though that second page is going to be almost entirely empty), but lots of small pages mean the translation tables have to be bigger to track where everything is, resulting in more overhead.

This is starting to reach the edges of my knowledge, but I believe what this is describing is that RISC-V chips and ARM chips have the ability for the OS to say to the hardware “let’s use bigger pages than normal, up to 64k”, and the Linux kernel is getting enhancements to actually use this functionality, which can come with performance improvements. The MMU can store fewer entries and rely on the OS less, doing more work directly, for example.

Bluesky’s more like an aspirationally decentralized platform, you can keep your own data on your own server and use your own domain name as a user name, but most of the rest of it is “centralized, but we’re designing it in such a way that we can open it up later.” Even then, though, it’s heavily influenced by the original idea of “let’s make something decentralized that Twitter can switch to once it’s worked out” which means that even when they do open things up, it’s likely that a lot of Bluesky will only be practical at “big tech company scale” to run yourself, whereas Mastodon or Lemmy you can just spin up on a server and it’ll be fine until you get a lot of users.

I as a human being have grown up and learned from experience and the experiences of previous humans that were documented or directly communicated to me. I can see no inherent difference with an artificial intelligence learning on the same data.

It’s a massive difference in scale. For one, before you even leave the womb you have millions of years of evolution shaping the initial structure of your brain. Then your “training” begins, but it’s infinitely richer than anything we’re giving to these LLMs. Sights, sounds, smells, feelings, so many that part of what your brain is learning is what it must ignore. You’re also benefitting from the interactivity of your environment, you can experiment with things and get feedback for what happens. As you get older and develop more skills, you can start integrating them together to do even more complex things, and the people around you will use their own incredible intelligence to specifically tailor your training to what you need as you learn and grow.

Meanwhile, an LLM is getting fed words, and learning how to predict the next word. It’s a pale shadow of the complex lives humans live. Words are one of the more powerful things we have for thinking and reasoning, so if you’re going to go all in on one skill, it’s a rich environment for learning and in theory the contents of all of humanity’s writing probably contains all the information necessary to recreate human intelligence, but our current technology doesn’t even come close to wringing every ounce of knowledge from the training sets.

The UN is supposed to be a toothless, executively dysfunctional institution, that’s a feature, not a bug. Its members are nations, whose entire purpose is to govern their regions of the planet. If the UN itself had the power to make nations do things, it wouldn’t be the United Nations, it’d be the One World Government, and its most powerful members absolutely do not want it to be that, so it isn’t.

It’s supposed to be an idealized, nonviolent representation of geopolitics that is always available to nations as a venue for civilized diplomacy. That’s why nuclear powers were given veto power: they effectively have veto power over the question of “should the human race continue existing” and the veto is basically a reflection of that. We want issues to get hashed out with words in the UN if possible, rather than in real life with weapons, and that means it must concede to the power dynamics that exist in real life. The good nations and the bad nations alike have to feel like they get as much control as they deserve, otherwise they take their balls and go home.

It’s frustrating to see the US or Russia or China vetoing perfectly good resolutions and everyone else just kind of going “eh, what can you do, they have vetoes,” but think through the alternative: everyone has enough and decides “no more veto powers.” The UN starts passing all the good resolutions. But the UN only has the power that member nations give it, so enforcement would have to mean some nations trying to impose their will on the ones that would’ve vetoed. Now we’ve traded bad vetoes in the UN for real-world conflict instead.

What that “get rid of the vetoes so the UN can get things done” impulse is actually driving at is “we should have a one world government that does good things,” which, yeah, that’d be great, but it’s obviously not happening any time soon. Both articles mention issues and reforms that are worthy of consideration, but the fundamental structure of the UN is always going to reflect the flaws of the world because it’s supposed to do that.

It’s all a stupid game. Israel killed some people, so Iran “had to” respond to save face, and then Israel had to do the same. Once everyone is satisfied that everyone knows both countries have very big and girthy missiles, they can finally back down from a war no one wants (hopefully).

Archive Team often uses the Internet Archive to share the things they save and obviously they have a shared goal of saving a copy of everything ever made, but they aren’t the same people. The Archive Team is a vigilante white hat hacker group (well, maybe a little bit grey), and running a Warrior basically means you’re volunteering to be part of their botnet. When a website is going to be shut down, they’ll whip together a script and push it out to the botnet to try to grab as much of the dying site as they can, and when there’s more downtime they have some other projects, like trying to brute force all those awful link shorteners so that when they inevitably die, people can still figure out where it should’ve pointed to.

OPML files really aren’t much more than a list of the feeds you’re subscribed to. Individual posts or articles aren’t in there. I would expect that importing a second OPML file would just add more subscriptions, but it’d be up to the reader app to decide what it does.

I think that joke’s been around for a while, but there is the Terry Pratchett line about how if you had a button with a sign next to it saying “pressing this button will end the world, do not touch,” the ink wouldn’t even have time to dry.

If you ask an LLM to help you with a legal brief, it’ll come up with a bunch of stuff for you, and some of it might even be right. But it’ll very likely do things like make up a case that doesn’t exist, or misrepresent a real case, and as has happened multiple times now, if you submit that work to a judge without a real lawyer checking it first, you’re going to have a bad time.

There’s a reason LLMs make stuff up like that, and it’s because they have been very, very narrowly trained when compared to a human. The training process is almost entirely getting good at predicting what words follow what other words, but humans get that and so much more. Babies aren’t just associating the sounds they hear, they’re also associating the things they see, the things they feel, and the signals their body is sending them. Babies are highly motivated to learn and predict the behavior of the humans around them, and as they get older and more advanced, they get rewarded for creating accurate models of the mental state of others, mastering abstract concepts, and doing things like make art or sing songs. Their brains are many times bigger than even the biggest LLM, their initial state has been primed for success by millions of years of evolution, and the training set is every moment of human life.

LLMs aren’t nearly at that level. That’s not to say what they do isn’t impressive, because it really is. They can also synthesize unrelated concepts together in a stunningly human way, even things that they’ve never been trained on specifically. They’ve picked up a lot of surprising nuance just from the text they’ve been fed, and it’s convincing enough to think that something magical is going on. But ultimately, they’ve been optimized to predict words, and that’s what they’re good at, and although they’ve clearly developed some impressive skills to accomplish that task, it’s not even close to human level. They spit out a bunch of nonsense when what they should be saying is “I have no idea how to write a legal document, you need a lawyer for that”, but that would require them to have a sense of their own capabilities, a sense of what they know and why they know it and where it all came from, knowledge of the consequences of their actions and a desire to avoid causing harm, and they don’t have that. And how could they? Their training didn’t include any of that, it was mostly about words.

One of the reasons LLMs seem so impressive is that human words are a reflection of the rich inner life of the person you’re talking to. You say something to a person, and your ideas are broken down and manipulated in an abstract manner in their head, then turned back into words forming a response which they say back to you. LLMs are piggybacking off of that a bit, by getting good at mimicking language they are able to hide that their heads are relatively empty. Spitting out a statistically likely answer to the question “as an AI, do you want to take over the world?” is very different from considering the ideas, forming an opinion about them, and responding with that opinion. LLMs aren’t just doing statistics, but you don’t have to go too far down that spectrum before the answers start seeming thoughtful.

In its complaint, The New York Times alleges that because the AI tools have been trained on its content, they sometimes provide verbatim copies of sections of Times reports.

OpenAI said in its response Monday that so-called “regurgitation” is a “rare bug,” the occurrence of which it is working to reduce.

“We also expect our users to act responsibly; intentionally manipulating our models to regurgitate is not an appropriate use of our technology and is against our terms of use,” OpenAI said.

The tech company also accused The Times of “intentionally” manipulating ChatGPT or cherry-picking the copycat examples it detailed in its complaint.

https://www.cnn.com/2024/01/08/tech/openai-responds-new-york-times-copyright-lawsuit/index.html

The thing is, it doesn’t really matter if you have to “manipulate” ChatGPT into spitting out training material word-for-word, the fact that it’s possible at all is proof that, intentionally or not, that material has been encoded into the model itself. That might still be fair use, but it’s a lot weaker than the original argument, which was that nothing of the original material really remains after training, it’s all synthesized and blended with everything else to create something entirely new that doesn’t replicate the original.