zlacker

This is perhaps the key bit of silliness:

"And this is where we run into the first bit of craziness. Amazon decided that they should model the Alexa app store after the iPhone app store. So there is a certification process to get your app into the store. But think about the difference: you are not uploading a binary file to the Alexa app store, you are simply registering an URL. So Amazon has no real control over your software. You could get an app approved, and then you could swap out the app for any other app, and the Certification team at Amazon would never know. They don’t control your code. Your code is not in their store, so they have no control over what you do. And yet they modeled this process after the iPhone store, where Apple does have control over your app."

But that doesn't get at how crazily broken the certification system. You have to read the quotes from the other developers to understand that.

>>lkrubn+(OP)
Amazon has just as much reason as Apple (or any platform provider) to want to restrict the types of apps that get published.

As for the effectiveness of that control, there's little practical difference between "simply registering an URL" and uploading a binary; both can load external content or be modified (in terms of what the user actually sees/experiences) almost entirely after the initial review. There will always be apps that try to abuse that possibility, and they usually get reported or re-evaluated, but it's probably a very small number of apps that will do that.

The point is to simply filter out a larger number of spammy, malicious, or otherwise offensive apps at the onset rather than spending more resources constantly monitoring everything after the fact. That said, it's unfortunate when the review process is as flawed as it seems to be in this case (poor communication, inconsistent policies, etc).

>>dperfe+I2
Apple has tools to see whether your app is loading external content and modifying itself, though. I think you can deceive them by making it seem as though it does so in an allowed manner (e.g. no code execution) during review, but they'll definitely know that your app has the theoretical capability.

>>dperfe+I2
There's a huge difference. An URL is an absolute black box, a binary that you have to give Apple for certification on each change and that has to call Apple APIs to do anything useful (like network) is not.

Apple will scan your App to check for basic violations, that's all automated. For instance is there a specific flag that you can initialize a socket with to listen in background. This is only allowed if your app is supposed to stream music in background or do some VoIP thing. So if you're app is not that and the software detects you have that flag set anywhere in the code no way you will get it through the certification. There's simply no possibility for such analysis with a URL.

>>lkrubn+(OP)
You are assuming that the only purpose of certification is to catch actively malicious developers. I can think of many other perfectly good reasons to have one: to make sure server response times are fast enough, to make sure it fits the guidelines for types of content they want in their ecosystem, to make sure it doesn't blatantly violate any trademarks, etc.

I don't think the concept of a certification process is the problem, just the implementation is terrible (compared to the Apple process which is merely "poor")

>>inflag+64
It sounds like you're assuming the majority of (iOS) App Store rejections deal with using unofficial/restricted APIs. My guess is that's relatively rare.

Most of the apps caught in the review process are probably those that fail other guidelines (usefulness, privacy concerns, illegal/inappropriate content, poor UX/quality, excessive crashing, etc). Most of those things can't be caught by automated means, and the content-related things can certainly be changed after initial review.

I've personally worked on several apps that significantly modify app behavior after being published - not so much to bypass any review requirements, but rather to adapt to changing business needs without waiting to publish a new release.

>>eli+Y4
You're definitely right that we don't actually know what checks they're prioritizing, but it's hard for me to believe they're actually doing any quality checking when this is the #1 skill you see in the app: http://i.imgur.com/Qp4Cv6k.png

>>eli+Y4
But that doesn't refute the parent's point: since the content at a URL is inherently mutable, they could judge that (at time of submission) some app is the type of content they want in their ecosystem, and then seconds after approval, it no longer is.

>>jc4p+o5
Most accurate software company name ever.

>>lkrubn+(OP)
I think part of the point is to control the NLU phrases, not the application.

>>inflag+64
Another interesting thing about iOS- they enforce code signing on the page level. You cannot create a RW page, write new code (from the network) into it, and then remap it as RX; every instruction needs to be approved by Apple. So changing behavior at runtime in a way that's subtle enough to pass certification is a lot harder than it would be on some hypothetical walled-garden Android (where you can, AFAIK, pull down a JNI extension over HTTP and load it.)

>>lkrubn+(OP)
I think it makes some sense. They know Apple won't just approve any app (malicious, useless, etc) so they only do the additional checks for the app that are specific to them.

>>jc4p+o5
The iOS app store was mired in fart apps in the early days. Many of them were very profitable. I think it's just a natural growing pain of any consumer-facing app store.

http://venturebeat.com/2008/12/23/iphone-fart-app-pulls-in-n...

>>SilasX+Q5
I would assume the terms of service prohibit that, which probably puts some liability on the developer.

>>inflag+64
This is kind of missing the whole point. Which is, as a developer, you can slip anything into the Apple appstore that you could have slipped into Alexa's, so the point the article is making about certification is silly. Of course there are more security implications in Apple's case, but the fact remains that numerous apps have done the bait and switch and were pulled after Apple discovered it much later.

>>lkrubn+(OP)
This is actually not an issue at all, because even with native apps you can still load content from the web which apple has no control over, which is similar to Alexa.

But the big deal that no one talks about is that Alexa is not compatible with EC2 backends, this is the most bizarre limitation I've ever seen, you can host An Alexa app on your own PC at home, but not on EC2.

>>mbdev+Jd
> Alexa is not compatible with EC2 backends

Source on this? I know last time I played with it, they weren't doing SNI (in 2015, what?) but I've never heard of it not being able to hit EC2 IPs.

>>inflag+64
There's a big exception to the automated checking. You can download new JavaScript code any time after your app is approved, as long as your app runs the code under the built-in WebKit. I don't think there is any automated checking of what your new code does - how could there be? It's just a violation of the terms of service if you change the app into something completely different:

> 3.3.2 An Application may not download or install executable code. Interpreted code may only be used in an Application if all scripts, code and interpreters are packaged in the Application and not downloaded. The only exception to the foregoing is scripts and code downloaded and run by Apple's built-in WebKit framework or JavascriptCore, provided that such scripts and code do not change the primary purpose of the Application by providing features or functionality that are inconsistent with the intended and advertised purpose of the Application as submitted to the App Store.

https://developer.apple.com/programs/ios/information/

http://info.meteor.com/blog/apple-hot-code-push-mobile

>>jdietr+3a
I remember those days vividly, what's different here is the Alexa Skills list is sorted alphabetically so this fart app is what every single person who enters the "store" sees first.

>>schlar+Xd
I had weeks longs mail conversation with the support and engineering team there.

And at the end they gave up and said that yeah its something deep in the implementation and I should use something else.

I don't want to post the mails but anyone can try it out !

>>SilasX+Q5
It could be that the client device validates a checksum against the approved list at the store before installing? Haven't tested if this is the case, just spitballing a mechanism that would allow for the control without the hosting.

>>mbdev+ef
This is incorrect - you can absolutely host in EC2. Our two skills are running off of a single EC2 micro instance.

There is one really weird thing where you can't use US West to do an Amazon Lambda passthrough to your server, but as far as I know EC2 instances in any region should work for Alexa to call out to.

>>galact+ch
I'm sure that the issue can be fixed so it might have been.

But I have the mail conversation here (10 people at least and dozen of back and forth).

This was 2 months ago and I gave up on it for this specific reason.

>>DDub+qf
https://github.com/substack/hyperboot

>>lkrubn+(OP)
Yes they could. They could have a daily check to see if the content at your URL has changed.

Also, you can similarly get around the iPhone cert process buy only having certain code run after a certain date, or when you trigger something remotely.

>>mbdev+Uh
Man, that's weird, I wonder what was different between our implementations. Our original POCs were about 4 months ago, and they were on EC2 at the time. We had plenty of other technical issues (don't get me started on SSL), but running on EC2 was never one of them.

That said, I'm curious how you managed to get an email conversation going with the team - the whole crux of the original article by Lawrence (and the forum thread which it cites) is that there's no way to have a direct conversation with anyone representing Alexa, so certification is a crapshoot.

>>galact+hq
I knew someone working at Amazon. That is why I'm reluctant to show the conversation or be more public about it at that time

>>SilasX+Q5
Right. Like I said, it doesn't do much for a malicious developer actively trying to subvert the process. That doesn't mean it's useless. I would guess most problematic apps are not malicious, but are an honest misunderstanding or disagreement about what level of quality is acceptable or what types of services are allowed.

A moderately clever developer could sneak something past the Apple app store review too. Wasn't there a flashlight app that included a secret wifi tethering tool?