TL;DR, Your mac is too old to run a select few features of Photoshop, but you can continue to run Photoshop in general just fine. Since Photoshop primarily runs off RAM, you won't be kicked off PS any time soon, and you will only need vRAM for those features it mentions. Also it looks like Flame, Tree and Picture Frame are just some of photoshop's filters, which require vRAM to run.
edit: i'm not sure whether you can update the driver and get more vRAM, i think RAM is solely a hardware thing so i dont know if a software update will help, but might as well take a look at that link in the error window and see if anything can be done? /edit
we have RAM, which is the temporary memory that allows a computer to open and run multiple programs at once. You've got 12GB of regular RAM which is plenty to deal with photoshop etc. But computers also have a graphics card, which handle graphics in video games, 3D programs and other hefty media programs. Graphics cards come with their own amount of RAM, called vRAM, which is dedicated to running these graphics smoothly. You has 256mb of vRAM, as it says under the "Graphics" section of your stats. Basically, Photoshop needs double that to run 3D and those couple of filters.
Unless you're a tech whiz, it's incredibly difficult (maybe impossible?) to upgrade the individual components of an iMac, so you're stuck with your 256mb for now - which is why I say your computer is too old to utilise the features Photoshop is yelling about. For context on how far macs have come, my late 2015 iMac has 1.5gb of vRAM (this is still tiny in the grand scheme of computers, but macs have always been lesser on the graphical front)
If you want to google and learn more about these things, a high quality graphics card (which macs DON'T have) is called a Dedicated GPU, and these usually have gigabytes of vRAM on them. the add-on alternative for macs is called an eGPU (or, external graphics), and only a select few cards are compatible with mac's software. Keep in mind the eGPU is like a tacked on card that isn't compatible with every app either, and may not work with Photoshop. It requires a lot of googling and research if this is something you're interested in.
