Comcast: Back to the House

Amazingly, despite overwhelming amounts of data from myself, and data from people in Oakland, San Francisco and Santa Clara all commenting that their network sees the same issues around the same time of day (late evenings), once again I get a call from Comcast saying they want to come out to my house and attach things to the network here. They truly are fixated with the modem & the cable part of their network. And yet they cannot explain how something in my house could:

  1. Affect me only for bursts of time in the late evening, but be fine most of the rest of the time
  2. Affect other people here, and in various other cities in the bay area

Apparently, they just cannot see that this is probably impacting many of their customers. I have spoken to many here who see the issue but don’t have the patience to deal with Comcast’s totally abysmal customer support system. Even when it gets escalated, it appears they cannot get past it being localized to one house.

If they don’t want to look at the data in their routers, perhaps they should try calling some of their customers in the bay area and asking whether they use the network in the late evening regularly, and whether they have seen any issues with it.

Here’s the latest data I sent them:

January 16, 2017 (averaging 37% packet loss in the test):

January 17, 2017 (averaging 18% packet loss in the test):

Also from January 17, the trace showing losses at Sunnyvale again, and from nodes beyond it (likely dropped at Sunnyvale too – something they’ve always said was not happening):

I get it that their network can get loaded; I work in an industry that sees how much data traffic increases month on month as people use more and more high bandwidth services. But Comcast’s only role here is to provide the pipe to the Internet and keep it from being overloaded. I get it that there are huge swings in traffic volume across a 24 hour period; I don’t necessarily expect to see the full bandwidth during peak hours (right now, at 11am on a weekday, for reference, I am seeing about 30Mbps), but dropping to under 1Mbps is unacceptable. And the times when the packet losses are so great that my router decides the Internet is inaccessible are completely unacceptable. Personally, I would say once it drops below 50% of the bandwidth I’m paying for, that is a problem; I suspect they have a lower percentage in mind, but I doubt it is as low as 1%. If they do consider 1% of the contract bandwidth to be acceptable, perhaps the FCC should take a closer look at the service Comcast provides.

Comcast: Not Resolved

It is holiday season so I wasn’t expecting much to happen with the Comcast issue (and to be honest, I haven’t been online at 10pm as much either to notice it), but on December 30th I did get a call from Mark N. in Executive Customer Relations updating me on the fact that the engineering team had not been able to find anything, and telling me they were closing the ticket. He also stated if I saw the issue again that I should let him know and he would re-open it.

At the time, I commented that while I had not been online at the affected hour (10pm Pacific for those new to this thread) to be able to update him on my experience, I felt it was unlikely to have changed given that it has been ongoing for over a year now.

Then, on January 1st, I happened to be online and notice the typical stalls in video streaming and poor performance in Twitter and Facebook with images & videos not loading. I ran some speed tests and traces too, and got these:

Speedtest results

Notice the chart for the download starts high, and then drops fast to a very low level. That is a bit different to previous results (although this was 10:35pm too, and things were starting to improve, especially the video stream). The average is also much higher than it is at the peak of the problem (where I typically see under 1 Mbps download). But, at other times of the day I see 50Mbps or more for the download speeds, and a nice flat intra-test chart.

Traceroute

The trace shows the packet losses at Sunnyvale though, same as always. I suspect that this is the cause of the fast drop in the speed test too as the TCP window size adapts down to cope with the higher than ideal packet loss.

The burst also explains why my command line speed test script didn’t see anything even when the GUI version was showing degradation. I suspect the command line tool is using only a small file, whereas the GUI version I know adapts how much it downloads to make sure it gets a good average.

Another Report

Tonight, I happened to be on Twitter and noticed this tweet about another Comcast customer in the SF Bay Area having issues, and pointing at the Sunnyvale router:

When I reached out to @Pixel, he got back to me with the comment that he sees issues almost every night too around 10pm ± 1 hour. And he is in Santa Clara. So, now I have reports of the same performance problems from Alameda, Oakland, San Francisco and Santa Clara. But still I get this from Comcast support:

I seriously think that Comcast needs to invest in some training for their support people. When Frank Eliason was running the Twitter support team it seemed to be staffed with people who were on the top of their game. Sadly, that doesn’t seem to be the case any longer.

The same support person however did let me know that the ticket (ESL02794458) had been closed with the comment that it was resolved. Let me be crystal clear now:

I have never stated that the issue was resolved.

The request to close the ticket came from Comcast because they were unable to find anything. As I mentioned above, my response to that was that I felt it was unlikely to have changed since it had been the same for over a year, but I had not been home at the right time to be able to confirm or deny it was resolved. That is most definitely not the same as me reporting the issue as resolved. Furthermore, for the record:

The issue is not resolved.

And, as should be apparent to the people at Comcast by now, I am not the only customer affected by this. I strongly suggest they get their act together and find out what is causing these performance issues. And I would also suggest that perhaps rather than sitting in the Hayward head end trying to find ways to pin the blame on my modem/router/Wi-Fi, they take a trip to the Sunnyvale router’s location and have a look at what is happening there. It seems too much of a coincidence that that one node always shows high packet loss rates when there are performance issues, and not when everything is working OK.

Not Just Traceroute & Speedtest

One more thing I’ve heard is that these tools might not show problems when the rest of the network performance is OK. Let me be very clear on this for the folks at Comcast too:

I only run Speedtest and traceroute when I see other things failing.

More specifically, the applications I have seen failing are:

  • Video streaming from Amazon and Netflix (stalling, dropping down to the lowest quality stream, and even failing to load the catalog on our Roku box sometimes)
  • Media loading in Facebook & Twitter (text loads, but no images, including avatars/profile pics in many cases)
  • Web pages timing out on loading (especially more complex sites like the NY Times, the Washington Post or even the Amazon home page)
  • Video calling (Google Hangouts, FaceTime etc) disconnecting repeatedly

Only when I see something like that failing do I think to run speed tests or traceroute on my home network.

Anti-Charter School Nonsense

I heard an ad for this clearly political campaign a few weeks ago, but tonight a postcard arrived in the mail from them. The claims being made are, in my opinion, exaggerated at best, but more often just false. 

I don’t have a lot of experience with either traditional or charter schools yet, but in researching schools for our five year old it was clear that the charter options near here had better results, and spent more of their money in their classrooms. 

Open to All

One of the falsehoods put forward in this postcard & on the associated website is that charters are not open to any students. Instead, they claim charters are discriminating to select stronger students. That was certainly not the case for the charter we selected for our now kindergartener. The selection process was explained to us in writing and at the information night; it was a lottery with priority being given to siblings of existing students. In fact, we had less information about the AUSD process, and had to take time off work (something that is not easy for all parents to do) to attend an in-person meeting at the AUSD offices (which was, frankly, a total waste of my time & could easily have been done online, or via the mail).

It appears that have been a few questionable charters, but that is as much the responsibility of the school districts that are meant to provide oversight of those schools, as it is of the schools themselves. I do agree that having charters overseen by their local district is a mistake, and was bound to lead to cases where that oversight was lacking. It was also bound to set up awkward situations, such as was demonstrated in Alameda where several school board members were decidedly anti-charter on principle. Moving the oversight to a central, state level body seems to make far more sense to me. 

Any charter that is discriminating in enrollment is already breaking the law. Proper oversight would catch that sooner & could have it addressed. 

For Profit Schools

As their domain name suggests, another blatant falsehood being pushed by this campaign is that charter schools are being set up for profit, and backed by “billionaires” for their own personal gains. 

The billionaires concerned, being successful business people & investors, might simply be horrified by the amount of money school districts are spending on overheads. By my estimates, roughly 25% of AUSD’s annual expenses goes towards non-teaching salaries and benefits (about 50% goes towards teaching salaries and benefits). Backing charter schools, which tend to have lower overheads, means more of the money will make it to the classroom. 

If they were looking to get into the business of for profit schools, starting a private school would seem to be a better choice than starting a charter. Perhaps they just care about improving the standards of education in California, which, when I compare it to my UK education is sadly lacking, especially in the sciences. 

Standards 

On that subject, one of the attractions of the charter we picked was that it claimed to be able to keep kids who have attended preschool & have basically covered the kindergarten level work already, interested by having different levels within the same classroom, or even by mixing K & 1 groups based on ability. I have yet to see that happen, but I am certainly going to be asking very soon if we don’t see evidence of it.

Ours also had capped classroom sizes & a full time teaching assistant in each kindergarten classroom in addition to the teacher. All of which leads to better standards. Right now, our five year old has an 11:1 teacher to child ratio.

AUSD & Charter Schools

I was sent an interesting email yesterday that mentioned that an Alameda Unified School District board member, in fact the board President, had made some pretty scathing comments about how she feels about California’s Charter Schools generally. The comments were made as part of the discussions surrounding the renewal of the charter for a school that has moved to Oakland (because it was unable to find a suitable space it could afford in Alameda, but is still authorized by AUSD). The renewal was approved, in a 3-2 split vote, with the president actually voting in favor of the renewal, which makes the comments even more odd.

What Ms Kahn said was:

I oppose Charter Schools generally because I think they’re predatory. I think they’re dishonest. I think they don’t give a hoot for the community that they’re in. I think their self-interest overrides everything that they do. I think they put themselves forward as a free, on the public dime, option for parents that would like to have, remove themselves from a public school into a private school, but they can do that for nothing by buying into a charter.

Obviously, Ms Kahn is entitled to her opinion on whether charter schools are a good idea or not, but as she herself said in an interview with The Alamedan prior to her election, “charter schools are here to stay.” She went on to say “like it or not, by law the district has to cooperate in the development of charters, while exercising oversight to guarantee that they are delivering what they promised.”

Predatory & Dishonest

Her comments at the board meeting on November 10, 2015 do not seem consistent with those pre-election statements. Furthermore, it is disingenuous to label charter schools generally as “predatory” and “dishonest.” From my own research, the charter schools in Alameda are committed to providing a quality education to their students, and are popular with both students and their parents. Like it or not, they are working (and, at least in the case of the Alameda ones, they seem to be working better than most of the district’s schools).

Private Education

I also found it personally offensive that Ms Kahn would imply that parents looking at charter schools are trying to get a private education on the “public dime.” Like most parents, I am looking for the best possible education for my children. If that is a charter school, then so be it. If it is a private school, then we will certainly consider it. I certainly can’t see anything that would suggest that AUSD’s schools stand out as being excellent by any measure! I can also say categorically that the charter schools we have looked at are nothing at all like private schools (I attended private schools from age 7 up, so I have some experience there).

Perhaps, if the AUSD board president feels that private schools offer a better education than the district’s schools (they almost certainly do), she should make it a priority to fix that disparity rather than making disparaging comments about parents looking at charter schools instead of district ones. Furthermore, if she feels that charter schools are able to deliver a better quality education on a public education budget, then perhaps she should look into how they can do that, and what can be improved in the district’s schools to achieve the same results. (To be honest, I don’t think the charter schools are delivering as much as they could

Bylaws

AUSD board members should probably also be aware of the section of the AUSD Board Bylaws that states:

Board members shall hold the education of students above any partisan principle, group interest, or personal interest.

(From BB 9200 – Board Bylaws – Limit of Board Member Authority)

Any attempt by AUSD board members to block a charter school’s existence based purely on a personal opinion about whether charter schools in general should exist, would seem to be contrary to that bylaw (and possibly to California state law). It also seems to me that the AUSD should be celebrating successful schools in its district, whether they are directly run by the district or merely overseen as a charter.

Ferry Gets Half-Baked Clipper Support

The recent addition of Clipper card readers at the ferry terminals in Alameda and SF looked like at long last the ferry service was going to get Clipper support so books of paper tickets would no longer be needed. And so it is, on October 1st.

Unfortunately, as with many aspects of the SF Bay area public transit system, those in charge of planning the addition of Clipper apparently don't use the service at all, or even listen to people who do.

Continue reading

Iconic Sea Nettles

Iconic Sea NettlesYou knew, given all the other photos from the Monterey Bay Aquarium, that at some point I had to post the iconic blue & orange sea nettle photo. Well, here it is.

For anybody who’s been down there, you’ll know just how hard it is to get a good shot of these guys. The room is dark, and people are pushing to get to the front and be right up close to the jellies. Oh, and did I mention that they’re constantly moving? Well, they are, not fast, but fast enough to make it that little bit harder to catch them well.

BART

For those not living in the San Francisco area, BART is a regional train service here that connects many of the East Bay cities with San Francisco and the cities on the SF peninsula down as far as the international airport (although connecting to the airport was something of an afterthought that was only added in the time I’ve been living here).

With the ending of the shuttle I was taking from SF to San Bruno, where Devicescape‘s headquarters is located, my only option from Embarcadero to San Bruno is BART. Now, over three months into that commute, I have some observations about BART that as an infrequent user I hadn’t really noticed all of.

Noisy
OK, it would be a lie to say that I had not noticed the screeching BART trains make through the tunnels as an infrequent user of the service, but now I am on their trains for an hour each day (30 minutes each way), the screeching is unbelievable. I’ve never been on a train service anywhere else in the world that is so loud. Most of the time a conversation with the person sitting next to you is impossible. My noise canceling headphones only managed a small reduction in the noise level.

Dirty
Choosing cloth carpets for a commuter train in a place where rain is not that uncommon has to rank up there as one of the most stupid decisions ever made. And now the cars have been in service for many years, those carpets are stained and disgusting. Granted, there are a few cars with new hard floors in use, but some of them are labeled as “demonstration” cars, and they are clearly getting old themselves, so unless the demonstration period was measured in decades, it looks as though no plan to upgrade the remaining cars was ever agreed upon.

Cloth seats were also perhaps not the best choice, but at least that is more common on trains that serve cities as far apart as BART does. The problem is that most train services clean them, or replace them when they become too soiled. Apparently not BART; frequently I see seats that are so badly stained I would rather stand than go near them. And that’s just what I can see. A recent report found all kinds of unpleasant stuff hiding in the BART seating.

No Food & Drink
Even more bizarre for a regional train service with journey times of 30 minutes to over an hour, food & drink is banned. They even interrupt useful next train information announcements with warnings about large fines for consuming food or drink on board.

Most train services are happy to have concession stands and/or vending machines within their stations selling food & drink because they get much needed revenue from such stands. Some train systems even have carts being pushed through the trains, or specific cars where food & drink is sold. Not BART.

The reason? They want to keep the trains clean. Clearly that’s not working. They’d be better off selling food & drink in the stations & actually cleaning the trains.

Inconsistent Announcements
I mentioned in the last section that the no food & drink announcements frequently cut off information announcements about the next train (sometimes even the one about the train currently at the station). It is not just the food & drink ones though. BART has a plethora of poor audio quality (often complete with office noise in the background) announcements that are played from the central control centre and will always override the station specific train information.

Bad announcements are not just limited to the stations though. On board announcements are made by the operator. Some are really good at this, others not so good. Either way, it appears that there is no script for what should be said. I often catch a train in the mornings where the operator announces the destination station name at each stop – very disconcerting if you weren’t paying attention. Others give a little more detail, like “This is Daly City; Millbrae train,” which makes it much clearer.

BART frequently stops mid-tunnel, especially the evening trains for some reason, and once again the operator information varies from total silence to a clear reason. This sort of thing should be consistently clear.

Escalators & Ticket Barriers
Everybody understands that escalators & elevators break down, but not daily. Embarcadero station, it seems, has its escalators out of service more often than in service these days. Shopping malls manage to keep their escalators running most of the time, why is it so hard for BART?

Even more annoying, when the escalator is out of service, they chain them off, forcing everybody into the narrow staircase next to it. An escalator can still be used as stairs when it is not running, but it seems BART is unaware of this.

Delays
Delays on train services (at least outside of Japan) are not uncommon, but BART seems to have them more frequently than I would expect, and without reason. We will frequently stop between stations for minutes at a time with no more explanation than “there is a traffic jam ahead” (I kid not, that was an actual operator announcement from a few weeks ago).

Indicator Boards
Finally, the overhead indicator boards that are on each platform have 4 lines of text on them than could be used to continually advise about the time for the next train. Instead, BART has chosen to fill them with more of their pointless announcements most of the time, or have them all show just the current time (something I can get from my watch or cellphone).

There is enough space on these boards that they could easily reserve one or two lines for next train information all the time, and scroll their other messages in the other portion of the screen. The current usage of these displays is just another indication that BART management knows very little about running a train service.

Too Rough

Too RoughAs you can see from the photo, the weather out on the bay tonight was far too rough for the ferry service to be running. With rough seas like that, what captain would want to risk taking their high tech catamaran ferry out of the safety of its dock.

Seriously, tonight’s ferry services from San Francisco to Alameda/Oakland was canceled. The service website said this:

Due to extreme weather conditions, ferry service is canceled for the remainder of today.

Does it look that bad? And remember that they ran the boats quite happily through the worst of the weather this morning.

Bus Bridge

As a replacement for the commute ferries, a bus service was operating, but the bus carried around half the normal load of the evening ferries, and was timed to depart from the ferry building 10-15 minutes before the normal ferry departure time. If I was being cynical, I’d say that was by design to encourage people to use alternatives and lighten the load on the buses!

Bay Ship

Bay ShipMore photos of the sunset over the city tonight, only tonight I had the 20D with me, so the photos are sharper! Also got this reflection of one of Bay Ship’s sheds reflecting in the almost totally still water it is built over.

The ship yard was still busy, with crews working on the luxury motor yacht Pangaea (which has been in the yard pretty much the whole year – and seems to be undergoing a major refit) as well as some other vessels they have up on shore or in the floating dry dock.