Access April with Our Bridges Resource Library
This month we take a deep dive into what may be the most important need of blind and low-vision people in our current Information Age: ACCESS. Accessible assistive technology (AAT) is great, but it’s only a tool to access the materials we need. We need to know how to use our technology, determine whether or not something is accessible, and advocate for the access we need if necessary.
- April 2: Accessibility of the New “IRS Free File” Tools
- April 9: Chris’ Favorite JAWS Power Moves
- April 16: Awesome JAWS Access Tools: Convenient OCR and Voice Assistant
- April 23: JAWS Picture Smart AI Feature
- April 30: Chatting About ChatGPT
This week we dive into an exciting new feature in JAWS that gives users more access to the information on our screens than ever before!
Check Out This New JAWS Feature
If you’re running JAWS 2024, you may have noticed a new notification in your JAWS message center inviting you to try a new feature. Though this invitation is extended nearly every time JAWS does a major update, this one was particularly special. It’s called Picture Smart AI.
What is Picture Smart AI?
With Picture Smart AI, JAWS provides users the opportunity to unlock the power of artificial intelligence in describing graphic images. Yes, that’s right: when you confront an image, a button, or any other non-text object that JAWS cannot read, you can launch Picture Smart AI and immediately access descriptions of that object.
What information does Picture Smart AI provide?
When launched, Picture Smart AI generates three levels of description – each providing more detail than the one before it. Access to all three of these independently generated descriptions provides a rich source of information, but you may also focus on just one level if desired.
Perhaps the most empowering feature of Picture Smart AI is the ability to copy and paste the descriptions generated. This way, you can have access to the information anytime you want.
How does Picture Smart AI work with alt text?
Picture Smart AI works with graphic images, including those that have alt text descriptions already. This can be useful because alt text descriptions may or may not provide sufficient detail for the task at hand.
Bonus: Educators can use Picture Smart AI, too! Our Director, Carlton Walker, has used Picture Smart AI descriptions to help her draft alt text descriptions for student’s materials. While Carlton remains the drafter of the final alt text, the Picture Smart AI descriptions provide a good foundation upon which to build accurate and descriptive alt text.
The Smarts Behind Picture Smart AI
Picture Smart AI takes advantage of two popular language generators, Chat GPT from Open AI and Gemini from Google, to create its descriptions. You may already be familiar with similar technology used for descriptions in the Be My AI feature of the Be My Eyes app. What makes Picture Smart unique is its ability to be used in real-time while on other screens. Let us count the ways.
Playing Around with Picture Smart
Chris Nusbaum, our project coordinator, has been fascinated by the seemingly endless possibilities of AI ever since he downloaded the ChatGPT app (more on that in our next Transition Tip). So, it comes as no surprise that he couldn’t stop throwing pictures at this new JAWS feature to see what it could do once he enabled it. Still, the experiment was only a cursory look, and more will certainly be discovered. Here are 2 examples:
Describing a graphic
First, Chris ran the graphic we post to our social media along with our Transition Tips through Picture Smart AI to see how it does with a file saved to his computer. Here’s the procedure he used, followed by the result—we think the description from ChatGPT captures our mission very well.
Press JAWS key + space, wait for the click, then let go and press P to launch Picture Smart.
Press F or shift F, depending on if you want a description from one or both services. I like both for reasons you will soon read, so I pressed shift F.
Results from Gemini
The image is a bridge with two yellow ends labeled “High School” and “College Jobs”. The bridge is supported by two red towers labeled “Help Desk”. The background is white with a black heading “Transition Tip Tuesday”.
Results from ChatGPT
The image features a diagram that resembles a suspension bridge. The text “Transition Tip Tuesday” is displayed at the top. The bridge itself is labeled “HELP DESK” at the center, with the left side of the bridge labeled “High School” and the right side labeled “College Jobs.” The bridge structure is colored in blue with red vertical suspension cables holding up the roadway, which suggests metaphorically supporting individuals as they transition from high school through a help desk (maybe symbolizing guidance or assistance) toward college or jobs. The vertical cables could represent different services or supports provided. The overall message seems to be about how a help desk or support system can assist in the transition from high school to further education or employment.
Using Picture Smart AI with a video
Chris also opened a recording of a Zoom meeting he recently hosted and asked Picture Smart to describe the computer screen using “c” for control while the video played. This resulted in a description of the person who was talking at the time. This could mean that Picture Smart can also read presentations on the screen of a Zoom meeting, but Chris has not yet tried that.
More Information
From Freedom Scientific
For more details, please check out New and Improved Features in JAWS web page. Freedom Scientific, the makers of JAWS, also discuss Picture Smart AI extensively in the March episode of their podcast, followed up by an Open Line episode in which users share how they use the feature.
From JAWS users
A great discussion of Picture Smart AI from a user’s perspective, including a quick demo and some real-world professional and personal uses for it, was recorded in Episode 269 of Jonathan Mosen’s Living Blindfully podcast (the segment on Picture Smart starts at 1:22).
Note from Chris: I generally recommend Jonathan Mosen as a great resource on all things technology and accessibility. His podcast is a wealth of information for anyone wanting to know more about successfully living with blindness/low vision. You can subscribe to Living Blindfully wherever you get your podcast.
We Are Here to Help
If you need help experimenting with Picture Smart or just want to share your latest discovery, contact our free Bridges Helpdesk anytime!
We encourage all who want to make the most efficient use of JAWS to give these features a try. If you have questions or need a one-on-one training session, contact our free Bridges Helpdesk anytime.
Please check out the Bridges Technical Assistance Center Resource Library today!
Contact us
Follow the Bridges Helpdesk Facebook page for more transition tips, and please contact the Bridges Technical Assistance Center’s Free Helpdesk for Maryland Blind/Low Vision Transition Students, Families, and Educators anytime using:
- Our Accessible web form
- Email: Helpdesk@IMAGEmd.org
- Text or Leave a Voice mail message: (410) 357-1546
- Bridges Helpdesk Facebook page or Facebook Messenger
This unique project is being coordinated through The IMAGE Center of Maryland, a center for independent living in Towson, and it is funded by a grant from the Maryland Department of Education Division of Special Education/Early Intervention Services.