hollymath: Selfie: white person, three-quarter profile, smiling, brown hair shaved on the side we can see, chin-length on the other (Default)
[personal profile] hollymath
I said I'd try to get around to writing up some details of the museum thing before I forget them all, and I've got a little bit of time and energy before my day starts getting hectic, so!

I found out about this because lovely Emma who I knew from her being volunteering coordinator at MOSI when I did the course there a couple of years ago got in touch with me and said she's now working at Manchester Museum and one of the senior conservators there was looking for visually impaired people to tell them what we thought of some new stuff they're working on and did I want to come in for a chat about it.

I was actually just giving up one of my volunteering roles at that time and trying harder on the job hunt, but I couldn't resist this combination of museums and accessibility consulting that I learned a couple of years ago is an intersection I really love and have a lot of things to say about! And anyway, it wouldn't hurt to just come in and chat, right? That didn't commit me to anything. And I'd told Emma I was busy. So it'd be fine!

So I met Sam, a Senior Conservator, and Chris, who is working on this haptic thing. I only vaguely remembered what the word "haptic" meant at this point so if you're in the same state you can have the benefit of looking it up which I did not when I was shaking hands with people. "Haptic" means elating to the sense of touch, in particular relating to the perception and manipulation of objects using the senses of touch and proprioception. (Proprioception is your sense of the relative position of your own parts of the body and strength of effort you're using to move them.) Haptic feedback devices create the illusion of substance and force within the virtual world.

In other words, it can feel very convincingly like you're holding something, turning it around in your hands, with a virtual object. It's quite a trip.

This is relevant to museums because museum objects can now be scanned and replicated to a really impressive degree of realism. Digital copies can be made of a mummy or a dinosaur bone or whatever, and that can be made into physical copies thanks to 3D printing, as well as the digital haptic stuff. Both of these provide models that can be handled in a way fragile museum objects never normally would be of course. And for blind people, this means museums can go from "boring rooms where you stand there and passively receive whatever information any companions might share with you but can't interact with anything on your own" to the kind of rich, multifaceted interaction sighted people can expect and enjoy.

So the first time I went to visit nothing was quite working -- not the haptic interface-y thing, nor the cat, which is a replica of an Egyptian cat mummy coffin that has been cleverly modeled...actually, rather than me try to describe it, here's a picture.

So hopefully there you can see that the right side of it has been made to look (and feel, because it's made of the same kind of wood I think) exactly as the artifact we have now. The left side was sculpted to look (and feel, the black lines are ridges in the wood) how it would have originally been created to look. And the person in that photo is wearing headphones because there are sensors in various bits of the cat that when you put your hand over them tell you about various aspects of it. If you touch the broken ear, it'll talk about the broken ear. If you touch the cat's back, you're able to feel the join where the four pieces of wood that make up this mummy case come together so neatly, and it'll tell you about that. It's like having a curator stand next to you as you observe the object, and as you explore different parts of it she can offer you relevant information you might be interested in. But it allows that independence blind people don't often get enough of: they're not at the mercy of anyone else to read the information off a little card, to tell them things they don't care about, or to move them away before they're done being interested in the thing, just to give a few commonly-occurring examples of the situation people are currently in.

Anyway, just having this and the haptics explained to me (I got to see and feel the cat but the sensors weren't working that day) was enough to get me excited. That was a few weeks ago; Monday I was invited to actually come in and try everything.

And meet my fellow volunteer, M, who was also joining us that day. Like I said, she reminded me a lot of me but in my early 20s, which she is. She's very enthusiastic about museums, has been to this one a lot as a kid and knew it very well. She has a guide dog; one of the first things we did was compare amounts of sight and we were sitting across a table so she said she could tell I had a red top on and dark hair and pale skin but she couldn't really see my face. And she's got no peripheral vision or depth perception. Sounds very like the level I was at before I started wearing glasses.

So we had a coffee and Sam and Chris told us a bit more about the work they're doing. They told us they both have a background in sculpture so they're very interested and knowledgeable about the features of different materials, not just "what's it made of" but "how that does that make it feel, how does it respond to different kinds of touch. It sounded like they have that awareness of how much data can be obtained from touching something that I am used to only thinking of as belonging to other blind people. Chris was talking too about how accustomed we are to interacting with the world now on flat planes of glass thanks to our smartphones and tablets and so on, and that a whole new kind of language and set of conventions has to be developed for the haptic 3D interfaces. These are people who've thought a lot about things both relevant and interesting to me, and it was really exciting.

Then we went to see the stuff! First that haptics, which has a website here with a lot of pictures of things I'd find it difficult to describe for you. You interact with it via a handle that's inside that roundish hole in the middle of the thing; you can see people putting their hands in there so even sighted people aren't seeing what their hand is actually doing (which I imagine makes the illusion of touching the ancient vase or whatever more effective, though that's just occurred to me so I don't know). The handle has a little button on the top you put the tip of your index finger on, and this is what vibrates when it comes into contact with the virtual stuff on the screen. You can kind of see that in this picture.

The screen is giving you images and sound, and it's very cleverly done so that first you're introduced to the conventions of the interface: there's a "matrix" on the "floor" of a circle split into eight wedges like pizza, with numerical, color and musical feedback for each wedge so you can tell where you are that way. You start out with familiar objects so you're only having to learn one thing at a time, which is how to use the machine, before you go on to using it to learn about objects. I really like this because it's so easy to get overloaded. My previous experience trying to make a tour of a museum exhibit for visually impaired people suffered from having to both convey the complex scientific content of the exhibition, and then also add the extra necessary layer of information on what the physical objects and space in the room were like. It was tiring for us to talk about that much and I'm sure it would've been tiring to try to absorb it all.

So it was nice that here the first screen starts with an apple and a plastic tube and a cardboard box. They're described the same way that museum objects would be, so the apple is a Cox's and it was invented in some year and so on. So when you've moved on to mummies or whatever, you know what to expect.

M and I kept taking turns, because of course only one person can use it at a time, chattering excitedly to each other (it's all been filmed and I'm sure it'll be great for whatever funding bid or PR it'll no doubt be used for) about things like "I got the ball inside the vase!" (the yellow bouncing ball is the screen's indicator of what your fingertip pointer is touching, so once you steer the ball inside something you get the noises and feeling of what the inside of the thing is like). You can see on the webpage the objects we were examining, and I think it's pretty great that M's favorite is from the British Museum, the "killer pig" (it was two meters high, apparently!) skull is in America...the virtual experience of course doesn't have to be limited to what is actually in the museum's physical collection (though of course lots of the stuff here is from Manchester Museum, including my favorite, the soul house).

Sam and Chris were surprised how quickly we took to the machine and were able to spin the objects around (you can look at them from all angles, and there's something cool about being able to see the bottom of a vase because you know that'd never happen in a real museum), we found out what happened when you hit them with the yellow ball (which is like tapping them with a fingertip. I did really like that kind of feedback: I thought the soul house looked like it was made of stone but when I tapped it it made totally the wrong kind of sound (I imagined) for that, and Sam confirmed that it's made of ceramic. And we could tell the hippo bowl was cracked before we could find the crack (I eventually did once I knew to look for it) because it made a really "dead" thunk! kind of sound, not as resonant as the soul house.

And then the cat was working too! So we each did that while the other was working on the haptics. When Sam sat me down in front of it, she said she had already found an annoying thing with the interface that she was going to change, and I was pleased she'd "shown her working" in terms of explaining that when she was developing it she thought a kind of "double tap" would be good if people wanted to stop the recording part way through, but it ended up just meaning that if you moved your hand slightly it'd cut out and you'd have to try hitting the cat again to get it to resume. There are always things that you think "oh, this seems like a good idea" and you try it and it's totally not. Especially when you're trying to imagine the experience or preferences of people with a sensory impairment you don't have.

They were delighted with all of our feedback. Of course we were mostly saying everything was great, which it was, but they also eagerly took in my suggestions about how to maybe improve the flow of information on the cat -- I said start with the basics of how big it is, what it's made of, because knowing that kind of thing helps people like me who can see a bit get a "head start" on processing the thing in front of me. And while I don't want to take away the freeform individual-exploration element of it, it'd be nice if there was some way to tell how many, if any, sensors I'd not tripped yet because I ended up stumbling across a few bits of info repeatedly while I was groping the cat to see if there was anything I hadn't heard about yet.

It was nice to be dealing with people at a high enough level that they could and did take the feedback on board, which was something that just wasn't possible when I was doing what I thought of as "the blind tour" at another museum, which was expected to be a small cog in a big, and otherwise-already-finished, machine, and it didn't really work to tack that on at the end with no resources or understanding. Much as it was a challenge to convey the complex exhibit content and the information blind people need to navigate a space and appreciate the objects in it, it wasn't nearly as hard as trying to explain to the curators and staff what kind of support they had to provide for this to work, and so it never really worked. Everyone had the best of intentions, but no one appreciated how big an ask they were giving to raw new recruits.

And then Sam took us to see the mummy store room, which was awesome but not at all relevant to the volunteering thing, just something she ended up talking about with M and we both jumped at the chance. It was interesting but since this is so long and like I said that was a bit random, I'll stop here and maybe come back to that another time.

Here's a video, about ten minutes long, of the haptic thing in action. It was hard to watch it without wanting to move my mouse around to do stuff! It's such an easy interface that I got used to it really quickly.

(no subject)

Date: 2017-09-15 10:49 am (UTC)
strange_complex: (Leela Ooh)
From: [personal profile] strange_complex
Wow, this all sounds really cool and amazing! I'm really glad they sought out your feedback and that you were able to give them input that they could actually make use of. I can see it all being a bit hit when it's all finished and in general use.

(no subject)

Date: 2017-09-15 01:55 pm (UTC)
barakta: (Default)
From: [personal profile] barakta
Thanks for writing this, sounds amazing. I will reread without a migraine cos many thoughts and feelz (good ones!). yay! Not surprised you were KNACKERED after thobut!

(no subject)

Date: 2017-09-15 02:27 pm (UTC)
jesse_the_k: (Braille Rubik's Cube)
From: [personal profile] jesse_the_k
Thanks for this extensive write up! What a brilliant, well-thought-out interface. It looks like these folks are aiming these tools at everybody, which makes it more likelier that blind ppl will encounter It.

It's neat that the interface acknowledges that many blind ppl have useful vision.

How wonderful to have respectful, high-level communication with the developers. Did they say what moved them to work on the project?

(no subject)

Date: 2017-09-16 07:48 am (UTC)
norfolkian: (Default)
From: [personal profile] norfolkian
Amazing and fascinating! As someone who works in the museums sector, I found this really interesting - thank you for sharing. :)

(no subject)

Date: 2017-09-17 02:33 pm (UTC)
silveradept: A kodama with a trombone. The trombone is playing music, even though it is held in a rest position (Default)
From: [personal profile] silveradept
That sounds like an amazing system and I hope the feedback helps to make it even better.

(no subject)

Date: 2018-02-06 10:17 pm (UTC)
kaberett: Trans symbol with Swiss Army knife tools at other positions around the central circle. (Default)
From: [personal profile] kaberett
Thank you so much for this post, which I have just scraped together the brain to read -- I am super excited about all the tech you've discussed, and the development processes!

Tangential note on "sighted people not being able to see what their hands are doing helping with Believing You're Handling The Object": yes, known psychological phenomenon, it's possible to give sighted people phantom sensations using a fake hand provided it's roughly where their hand should be and they can see it and not the one that's attached to them.


hollymath: Selfie: white person, three-quarter profile, smiling, brown hair shaved on the side we can see, chin-length on the other (Default)

April 2019

  1 2 3 4 5 6
7 8 9 1011 12 13
14 15 16 17 18 19 20
21 22 23 24252627

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags