Our domain was abruptly blocked by our registrar this morning. Our NOC team and myself tried to get in touch with them and they tell us "Contact our legal". Even I could not get in touch with anyone beyond their phone operator. The domain was restored, but as DNS takes time to restore, we are still facing issues. They later claimed there were abuse complaints about Zoho.com emails (which is our personal email service with millions of free and paid users). We received a total of 3 complaints from them and two of them have been acted upon and one is under investigation.
Once we dig our way out of this, we will find ways make sure no one takes down our domain again this way.
Zoho is fine as a service, but a domain suspension shouldn't cut tens of millions of people off from email.
What is the alternative?
Google is far too omnipresent to just ignore.
Was it really a block at the registrar level or was it a block at the DNS level, i.e., the registrar also ran DNS service and their DNS service refused to return responses for zoho.com domains?
At what layer or at which stage of the protocol can a registrar disrupt this and take a domain offline?
Anyway, as a wakeup call -- if you have a business critical domain name, you need to find (and use) a registrar that has a registry lock procedure for the TLD you're in. A registry lock means the registry won't process changes from your registrar unless you authorize them, which makes it a lot harder to change things on purpose, or by an attacker. I imagine abuse takedowns could still go through though -- but there will at least be more people who know you care about your domain.
While the fight against it is rather dire and no end will ever be in sight, I'll nonetheless never stop (tool assisted) fighting.
Anyway, @zoho.com addresses used by spammers started to pop up circa a month ago and increased rapidly in occurrence. As we use stopforumspam to report and track spammer info (and surely are not the single forum seeing those @zoho.com domains) you may got a few flags raised somewhere.
Not sure what caused this sudden (from our POV) attraction of spammers using zoho, you may want to look into some defense against this. While a full solution may not be achievable it's often enough to be faster than other providers, aka the tiger defense ;-)
ns1.vtitan.com
pdns90.ultradns.net
pdns90.ultradns.com
dns1.p03.nsone.net
dns2.p03.nsone.net
nds3.p03.nsone.net
(Of course, some caching resolvers ignore TTLs :( )
Terms:
ICANN: The organization responsible for coordinating the maintenance of the domain name system (among other things).
Registrar: A company authorized to update ICANN database on behalf of registrants. Google, GoDadddy, Enom, etc are registrars
Registrants: An entity that wants to register a domain name. In this case, Zoho is a registrant, but it could also be an individual. This is your role if you 'own' a domain.
Authoritative Name Server: A domain name server that is considered authoritative for a specific domain.
Stuff registrars can do (among other things):
1.) They can update the ICANN database to disable a domain completely[1]
2.) They can replace your authoritative name servers with their own or someone else's (ex: botnet domains being reassigned to a security company for dismantling via court order)[2]
3.) If the authoritative name servers for a domain are owned by the registrar, then the registrar can merely change the DNS entries themselves to point to something other than the domain owner's wishes.
[0] - https://en.wikipedia.org/wiki/ICANN
[1] - https://www.icann.org/resources/pages/epp-status-codes-2014-...
[2] - https://www.icann.org/en/system/files/files/guidance-domain-...
what in the world?
What is considered a reliable registrar in Europe?
I work in this industry and it's a very clear separation between bulk registrars and those that maintain fewer but high value domain names. The latter usually give you a personal contact person to call and work proactively to deal with threats to companies' domain names and trade marks. I don't think I have ever heard of a domain being abruptly suspended by such a registrar.
The cost is usually 5x-10x that of the cheapest registrars so there is naturally a balance to be struck, and as I work in this industry I might be a bit biased. However the damage when waiting on the TTL when registries update NS records sounds very substantial when they first suspend and later restore a domain name in what sound as a very reckless behavior.
If your domains are riding on a credit card, you potentially have a failure mode of "card was declined, my domain did not renew, everything is down."
If you have something with 40M customers I'd highly recommend going with the same domain registrars used by some of the Fortune 100 companies.
Seizing a domain at the registrar level, by court order, is also how the US government implements "seizure" of domains, if you've ever seen a torrent index site that has suddenly been replaced with a big scary FBI page (examples: https://www.google.com/search?q=this+domain+has+been+seized+... )
Google is a registrar themselves... Do you mean they use someone else for their own domains?
Domain Name: GOOGLE.COM
Registry Domain ID: 2138514_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.markmonitor.com
Registrar URL: http://www.markmonitor.com
Updated Date: 2018-02-21T18:36:40Z
Creation Date: 1997-09-15T04:00:00Z
Registry Expiry Date: 2020-09-14T04:00:00Z
Registrar: MarkMonitor Inc.
Registrar IANA ID: 292
Registrar Abuse Contact Email:
Privately I’m pretty happy with Namecheap, they never failed to provide the support I needed in a friendly and precise manner. For business purposes with high value domains MarkMonitor seems to be the industry leader.
Registrar: Google Inc.
Registrar IANA ID: 895
Registrar Abuse Contact Email: registrar-abuse@google.com
Registrar Abuse Contact Phone: +1.8772376466
Verify yourself at: https://www.iana.org/assignments/registrar-ids/registrar-ids...
Zoho is Zoho Corporation Private Limited IANA ID: 3803
Also, last I checked, unlocking a domain for transfer to another registrar required emailing Google/eNom. There's no interface for it. For a while the entire UI to choose to cancel a domain just disappeared as well.
I would not recommend Google Domains.
I heard a lot of good things about German INWX[1], even though French Gandi[2] is more popular and is the registrar of ycombinator.com (and was the registrar of reddit.com until recently, before they moved to MarkMonitor).
While email is getting harder to run yourself due to all the bad actors, with dozens of reasonable choices (plus the option to self-host like I do) you can hardly call it centralized.
> We recently detected activities on our servers where bot nets were used to create hundreds of thousands of e-mail accounts for the sending of spam e-mail. Although we take this as a compliment – somebody out there must be convinced our infrastructure is up for the job – we needed to find a solution to stop this abuse of our service, of course. We subsequently deployed a number of different CAPTCHA systems to help our servers identify bots during registration. However, spammers were able to circumvent all these solutions shortly after they were put in place. [...] We therefore decided to use Google’s CAPTCHA for the time being, because out of the set of solutions we tried thus far, this one seems to work best.
[1] https://userforum-en.mailbox.org/knowledge-base/article/goog...
I wish you the best of luck once you catch up with the CEO of Tierranet (or perhaps you already have!)
[1] - https://www.epag.de/en/
I always thought that Google has a huge competitive advantage here, because most people browse the web being logged into their Gmail accounts, and, therefore, as with Google Analytics and Google Adsense, Google knows that it's you who is viewing that page. It can then present extremely time-consuming CAPTCHAs to anonymous visitors, most of whom are likely to be bots or the spammers themselves.
I've always been a bit perplexed as to how registrar's are created. How could I become a registrar?
Any advise or resources to explore this very open question would be wonderful.
Cheers J
"Cloudflare Registrar is the highest level of registrar security. It protects your organization from domain hijacking with high-touch, on and off-line verification of any changes to your Registrar account. Cloudflare is an ICANN accredited registrar providing secure domain registration for high-profile domains."
I was dismayed to see that someone can literally send one email, get your personal info, and impact your company.
Very disappointed in namecheap.
The lack of decent options of domain registers for technical people that don't need their hand held and have decent security, while not being $$$$ enterprise options is depressing...
I use Uniregistry which has TOTP support and what seems to be a competent team, and a friend swears by AWS's Route53 domain registration, but more choices with actual good policies and aren't just a reseller would be welcome.
This event seems to have been triggered from abuse complaints - and involved the registrar not reaching out to the client in question.
Curiously enough, I had a very similar incident with Namecheap last week: an unsubstantiated email (without subpoena, judge's order, or even validation of who actually sent the email) - was sent to namecheap abuse /alleging/ (correct, no proof) trademark infringement.
Namecheap rolled over and provided all information to the third party - and didn't bother to inform me of the incident. The only way I found out was a menacing legal letter using the address that I have on file at namecheap.
If Namecheap doesn't respect due process (ie, requiring legal documents to turn over customer information) or customer privacy (Hi, we have just had to turn over information) - on a 10+ year customer, I'm not sure that you're in a better position than Terra.
Severely disappointed with you guys.
https://www.icann.org/resources/pages/accreditation-2012-02-...
The cheaper, and easier way, if you're looking to start selling domains with a lower barrier to entry (but less control over how much you pay/how you sell your domains) is to find a white-label reseller registrar.
In all that time of being perplexed, you never thought to do a simple Google search? https://www.google.com/search?q=how+registrar%27s+are+create...
Disclosure: we're building a registrar on top of Handshake. We can also help you claim "zoho" on Handshake for free if you're interested.
That being said, the proper way to report abuse to an ISP is to email the official point of contact for abuse associated with their IP netblock. In the case of Zoho, that contact info can be found here: https://bgp.he.net/AS2639#_whois
ARIN rules require that all IP netblock owners provide a valid point of contact for abuse issues. ARIN validates the points of contact annually. I believe that RIPE, APNIC and LACNIC have similar rules.
If an ISP doesn't act on the abuse after it has been reported to their abuse point of contact, then you have a legitimate complaint against them.
Spoke with lawyers and from what i was told in consultations there's basically nothing I can do about it.
TL;DR Namecheap will endanger your family and they give 0 fucks.
This company literally has 0 morals and doesn't care about making sure people are treated right. Also, good luck getting through their regular support. It's straight from a script with 0 deviations.
If you do whois lookups against the top 50 websites you'll see a lot of them use a small set of registrar's. But not all of them accept small businesses.
The gold standard for any enterprise is MarkMonitor. You can pick any other enterprise level service which would mean you don't resort to lowering yourself to begging on Twitter to find a contact at a pivotal service provider
This has damaged you beyond DNS propagation, I don't know how anybody in tech is going to take you seriously again without some serious action
Can you shoot me an email? ted [at] namecheap.com
I'm so bloody sick of helping Google train their self driving cars. I swear I'm going to have PTSD about stop signs and store fronts for the rest of my life.
> Once we dig our way out of this, we will find ways make sure no one takes down our domain again this way.
Would look forward to an official email with regard to what steps were taken to mitigate this going forward.
Thank you for responding to this quickly. I saw this just a little while ago; I use Zoho Writer and Show for presentations and team-based doc editing and I have for the last decade. If Zoho goes down, I'd be very much lost. Thanks for providing a great service for this many years and I hope it keeps going for many more.
- Why use the same domain for the free service, which is usually more prone to abuse?
- Zohocorp.com is hosted on GoDaddy. Why not move all your domains to a single company so that they value your business more and give you a better level of customer service?
I hope once this is all over, Zoho just shares their feedback and some advices that will help small businesses.
Anything which startups can use and is $$ ?
I really believe in running a lean business, but running lean means cutting the fat, not cutting out your muscles and tendons and running with a naked skeleton that is fragile.
A commonly recommend option here in HN was NameCheap. Earlier this year without any notice they modified our DNS servers completely taking down our SaaS product.
Why? Some migration script run incorrectly.
They offered me a random TLD for free for one year as compensation! I declined.
I did some work for a client in 2017 who was starting a cryptocurrency business. This involved buying a domain name for him to transfer to him later.
Well in 2018 there was some internal strife in his business that ended with a lawsuit being started. The opposing party started sending subpoenas to Namecheap asking for all information from 2018 onwards in relation to his account. What ended up happening was they released all of my information about my purchases, domains, personal information(anonymized credit card info, my actual physical address, information about my other unrelated clients domains, etc.)... going back to the start of my account.. several years worth of data prior to 2018. All clearly out of scope of the subpoena they were served.
Not only that, Namecheap never notified me of this.. in violation of their own privacy policy. They're supposed to notify their customers of the release of their information in relation to subpoenas by email or certified mail. Instead I found out much later from my previous client when he was given a copy of all of my information. And presumably his opposing parties in the crypto space were also given all of my information.
Seems kind of messed up to release all of that erroneously, without warning... especially to shady people in the crypto space.. you know, with people getting kidnapped over this stuff.
TL;DR Namecheap will drop your info, even if you paid to protect it as soon as they're given a single demand letter. And they won't stop at just giving up the info that's asked for (with 0 fight and 0 notification to you) there's a chance they'll release ALL of your account information.
> I've started deliberately getting them wrong - probably won't make a difference, but makes me feel a little better about it
I don't even have to try anymore to get them wrong on a regular basis. Now, I think it's now more like training Google users to make the same recognition errors as its self-driving cars than training the cars to do a better job.
"Email marketing software that drives sales. Create, send, and track email campaigns that help you build a strong customer base."
They don't have 40 million users. They have 40 million targets.
Of course they don't get many complaints. If you search for "zoho opt out", you get sent to a page with a HTTP 400 error.[2]
[1] https://www.zoho.com/campaigns [2] https://help.zoho.com/portal/kb/articles/what-does-email-opt...
I noticed Amazon use a UK domain for one of the four Route 53 nameservers they specify.
I can only fathom these shops, both management and the webdevs, have no idea how unprofessional their site looks to anyone that isn't using a vanilla ISP connection. And my experience is coming from using a single longstanding VPS address, not even a shared VPN.
A sensible scheme would allow a certain rate of login attempts per any IP before hassling a user, but Google is obviously more interested in getting their training data than making sure you don't lose customers!
Not sure if this is the same, but I once came across a website with a captcha where you had to rotate a dog so it stood upright, but it was lagging so bad that it would skip several frames, making it impossible to time the angle correctly. After several minutes of trying I gave up and went to a different website with an inferior service, but which did not waste my time.
Also, Zoho is among the most trustworthy companies list of mine. They don't do funny business with AI and targeted Ads with your data.
You try, you pay and you use the software. Traditional, no-nonsense business model. I respect both Apple & Zoho for doing this. Just because Apple has a platform to run ads (The App Store), it doesn't mean Apple is in the advertising business.
Arrival-Date: Thu, 30 Aug 2018 00:00:00 +0200 (CEST)
Final-Recipient: rfc822; abuse@zohocorp.com
Original-Recipient: rfc822;abuse@zohocorp.com
Action: failed
Status: 4.7.1
Remote-MTA: dns; mx2.zohocorp.com
Diagnostic-Code: smtp; 451 4.7.1 Greylisted, try again after some time
And that's without getting into why you have a filter on your abuse@ address to begin with.
Is the problem systematic?
Thank you for your notification, will check on this and block those who spam using our system. However please put up an email to abuse at zoho.com so it would help us provide clue to our investigations. Reg Rajasekar Zoho Abuse Monitoring Desk.
I would not say MarkMonitor is a tool for startups. It's a tool for organizations that would lose a lot if they lost a domain. I bet Zoho wishes they could go back in time and spend $10k to avoid this problem they had.
Incidentally, both links just pop up a sign-up form.
Also since it said "suspended for abuse complaint", I would almost immediately assume the Zoho just didn't properly handle abuse claims and its their fault.
Needless to say I have a incredibly low opinion about their "service" based on having used their mail product for almost a year (switched to google afterward).
This is a key error in their handling of their abuse@ address, it needs to be expected to receive spam.
a captcha where you had to rotate a dog so it stood upright
Thank you for sharing your story. It should serve as a warning to others who may need to audit their infrastructure.
"Upon the receipt of a valid criminal subpoena, unless the circumstances or subpoena warrant otherwise, Namecheap may promptly notify the customer whose information is sought via email or U.S. mail"
Two things seem unclear:
1) The phrase "unless the circumstances or subpoena warrant otherwise"
2) The use of "may" in "may promptly notify the customer". Why is that not "shall" or "must"?
You want Google to not know about you. You want to be a stranger to them. And you are complaining that they don't trust stranger, which you want to be, as much as someone they know?
Never use namecheap for anything important.
I almost has a domain frozen with namecheap after one warning. If I missed the warning email or checked my email after 24 hours they would have completely suspended my domain. I'm talking about a site with MILLIONS of visitors per month and ten thousands of posts per day, not some small blog.
I repeat don't use namecheap for any meaningful business, especially anything that is "enterprise"
I almost has a domain frozen with namecheap after one warning. If I missed the warning email or checked my email after 24 hours they would have completely suspended my domain. I'm talking about a site with MILLIONS of visitors per month and ten thousands of posts per day, not some small blog.
I run a forum site with MILLIONS of visitors and about 5,000 TB of traffic per month. Namecheap.com suddenly sent me a link warning that they will suspend my domain completely within 24 hours, if I did not delete two problem images (which were inappropriate/troublesome images but in the context of the forum posts, "a very poor attempt at humor"). I deleted the images and avoided being suspended, but the way they threatened to suspend my domain due to two images was ridiculous. If I missed the warning email or checked my email after 24 hours they would have completely suspended my domain. I'm talking about a site with MILLIONS of visitors per month and ten thousands of posts per day, not some small blog.
They may be suitable for some blog, but I can now say to NEVER use them for any enterprise site.
Having a personal contact at the registrar for example might sound unnecessary, but it means that a person at the registrar should know the company involved and the impact of the domain or domains before any serious action like suspension are made. In large and bulk like registrar this isn't the case and as such no one likely knew what Zoho.com was or how many users it would effect. It was likely just an other $10 annual fee among millions of other domains, and as such it is very easy to just suspend and forget and later try fix any issues if those are raised. Cheap and quick solution but very costly if the owner values the domain name above that of $10.
Usually most processes involve some form of capital investment and/or technical capability. Country specific TLD can either be easier or much much harder depending on which country.
* Not really everyone.
So if you cant afford something enterprise like MarkMonitor, and you don't want something super cheap $9.99 per year. What sort of good quality middle ground choices do we have?
In my experience, the opposite is true in both cases. Big registrars can’t afford any support costs since they prefer to squeeze the price down as far as possible, and therefore they prefer to simply lose or outright drop any customer in case of any and all problems. Conversely, small registrars may charge more, but have better (i.e. actually existing, and sometimes even dedicated and personal) support for when things go wrong, and have a vested interest in keeping you as a customer.
"Upon the receipt of a valid civil subpoena, Namecheap will promptly notify the customer whose information is sought via email or U.S. mail. If the circumstances do not amount to an emergency, Namecheap will not immediately produce the customer information sought by the subpoena and will provide the customer an opportunity to move to quash the subpoena in court. Namecheap reserves the right to charge an administration fee to the customer by charging the customer’s Namecheap account."
This has already happened with tor and Cloudflare, but at least that changed for the better recently (see https://www.zdnet.com/article/cloudflare-ends-captcha-challe...). In that case it was just one CDN using captchas to discriminate against a group of users, so that one change by the CDN could fix the issue. If too many random sites are independently blocking or slowing down anyone not logged into Google, then that'll turn the web into Google's web.
One possible solution is a proof of work for name registrations, similar to the Onion Name System [1]. There is a short talk by Jesse Victors that explains it nicely [2].
Thank you for bringing this up. It was due to our greylist setting for *@zohocorp.com domain, we have now excluded the greylist for abuse addresses. Please resend your complaint to our abuse address. Regards, Zoho Account and Abuse Monitoring Desk.
400 Bad Request in Firefox.
curl:
curl https://help.zoho.com
<html>
<head><title>400 Bad Request</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
</body>
</html>This is a huge oversight.