Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Study HN: Linear Algebra 1. We're on.
28 points by gruseom on Oct 1, 2011 | hide | past | favorite | 33 comments
A couple of weeks ago, a bunch of us decided to put our internet time to better use and (re-)learn linear algebra together (http://news.ycombinator.com/item?id=2993321). Most have the book by now, so... let's do this! If you missed that thread and want to join, please do so. There's nothing you have to do other than learn.

The book is "Linear Algebra And Its Applications" by Strang, 4th ed., ISBN 0030105676. We'll work through one chapter every two weeks. There are a lot of exercises. Let's discuss ones that are interesting, nontrivial or both. Questions and insights about the material are welcome. The first 5 chapters have Review Exercises at the end. Let's call those "mandatory" and discuss them at the end of the two weeks.

Housekeeping:

1. We'll do this on HN unless it becomes intractable or the landlord asks us to leave.

2. Put "Study HN: Linear Algebra 1" in the subject line. Then we can find posts like this: http://www.hnsearch.com/search#request/all&q=%22linear+algebra+1%22

3. If you want to be emailed when something relevant happens, email me at the address in my HN profile and I'll put you on the list.

Finally, not all of us have the above book. Some are using an earlier edition or a different text altogether. Others have the 4th edition, but in a cheaper Indian format that is sold as identical, but - warning! - is not. Presumably to thwart students from getting away with using the cheaper edition for courses, the publisher has tastelessly scrambled the order of the exercises. How, then, are we going refer to exercises? Let's experiment to see what works. In the meantime:

4. To avoid confusion, only reference exercises by number (e.g. "Ex. 10 of 1.2") when they are from the official book (the 4th edition, expensive non-Indian format). If you don't have that text, and want to play along, email me and we'll figure something out.

So, HN: what's interesting in the early sections of Chapter 1? I have a few things, but let's hear from others. How about we stick to 1.1, 1.2, and 1.3 at first - up to Gaussian elimination, but not yet matrices.



Just for what it's worth: I'm playing catchup on 1.1-1.3 tomorrow and Tuesday, and I'm sure others haven't started yet too, so feel free to keep posting here and we'll try to keep paying attention to the thread.


(it took me an hour to get through the first 9 in 1.2. I'll get there! but console yourself with the thought that you, whoever you are, are far from the worst at this in the group.)


Whether or not that's true, and I doubt that it is, we all know who is hands down the wittiest.

I'm stuck on 1.3.21. What the hell is "the 1,2,1 pattern or the -1,2,-1 pattern"? Does he mean a pattern in the columns? I see 1,2,1 but not -1,2,-1. Is this like one of those duck-rabbit optical illusions?

Edit: there is, however, a -1,2,-1 pattern in 1.3.24, and the system there is very similar to the one in 1.3.21.


Yep: mechanical_fish. :P


For me, I don't think we should do the calculations by hand just for the sake of doing it. It should be done only as a means for understanding the core concepts. That is I much prefer to do exercises where the understanding of core concepts is brought to the attention.

So the question is "What are the core concepts up to 1.3"? I'm sure singular vs. non-singular is one. Are pivots a core concept? I think open-ended questions in the 2nd half of 1.3 questions are good candidates for deeper understanding.


We probably don't need to overthink this too much. We're not giving out grades, so do as many exercises as you think are necessary.

What would be awesome would be, as you decide which exercises to do, if you could tell the rest of us why you chose the ones you did.

My maths suck, so I'm going to try to get through all of them.


I'm going to try to get through all of them

That's inspiring. I've been working through them all so far, albeit slowly. It's worth noting that later chapters have fewer exercises. The one I'm looking forward to most, Chapter 8, has less than a third as many as Chapter 1 does. Presumably they get less mechanical as well. I'd be content if I find enough time to "pay my dues" the slow way through the earlier chapters. When I was younger I used to scorn the mechanical stuff. Now I'm taking pleasure in it. We'll see if that lasts.

For those who don't intend to do all of them, here are some exercises I found interesting:

1.2: 3, 10, 11, 14, 15, 23

1.3: 4, 14, 17, 18 (that's about as far as I've got so far)

(Also, see my comments above if you don't have a way to look up those exercises by number - and especially if you have the Indian edition.)


OK, these were interesting. Are we ready to share thoughts on them?

For 1.2.10, I got a single equation involving the three unknowns--does that sound right?


Let's talk in detail about 1.2 and hold off on 1.3 until tptacek confirms. Re 1.2.10, I think that's right. Because the x-coords grow by the same amount as you go from one point to the next, the y-coords must as well, giving y3-y2 = y2-y1.


Good. That's a really geometric understanding, which is helpful. I saw it as somewhat analogous to the back-substitution after lower-triangulation; using y=mx + b and starting with y_0 = b, then y_1 = m + y_0.


Yes, this is correct. (Though I haven't looked to see which problem this is, I can pretty much tell it's that one.)

If we're not through 1.3 yet I'm not far behind! Will try to plug away on my 1.2 problems in between bugfixes.


I'm glad to hear that. I've been hoping that a group pace would emerge. If it's slower than we planned, that's ok as long as it still overcomes friction.


> For me, I don't think we should do the calculations by hand just for the sake of doing it. It should be done only as a means for understanding the core concepts.

That's always been my take on the reason for doing hand calculations, but sometimes it takes much longer than I expect to really grasp matters. I've seen in really well-written math books some questions bo a much better job of leading you to discover concepts "on your own" than the teaching material.

One of my favourite methods to learn is to write code to embody the lessons learnt. There's no better way to learn a concept than to develop it through to the end, imo.


It would be so much fun to work through the book by writing code to do all the interesting stuff. I've been dreaming about doing just this in K; I've wanted to properly learn K for years and this is a dream domain for it. Silentbicycle, who knows a lot about K, is planning to join in this group at some point too. But alas, I doubt I'll be able to do it. My main project takes up pretty much all my hacking energy and I won't allow anything to detract from that. I've never done side projects well. :(


Are pivots a core concept?

Good question. They don't feel that way to me. They seem more like the implementation detail of an algorithm (Gaussian elimination). Is the algorithm itself a core concept? Hard to say no to that, I guess, but I wonder.

What about what Strang calls the "row picture" vs. the "column picture"? There you have two fundamentally different geometric interpretations of linear equations. That was a revelation to me. When I studied linear algebra a long time ago, I don't remember encountering the column interpretation. Perhaps I did and didn't pay attention; I was more interested in symbolic manipulation at the time, whereas now I want a sense of what the symbols mean geometrically.


Amusingly, I'm in the entirely opposite boat. I managed to get an entire Ph.D. in solid state physics without taking an actual course in linear algebra (a procedure that I do not recommend, but everyone is young and foolish once). So I've got some idea of what vector spaces are about geometrically -- otherwise, graduate quantum mechanics would have been impossible to navigate -- but I'm really bad at manipulating the actual symbols.

And, incidentally, I'm planning to attempt most of the problems. I've cherry-picked my way through this subject before, and I was ultimately displeased with the results. They say that the problem with an autodidact is that he has a fool for a teacher, and with linear algebra I learned that the hard way. Fortunately, now they've invented Open Courseware, Amazon.com, and HN, so I'll give the fool another chance.


I propose that we explicitly mention any exercises that we stumble over. Yeah it may make us look like idiots but we're destined to look like idiots anyway, and there's a major upside: we're going to stumble over different things, which means we can provide good explanations to one another, and learn better.

I have a few examples of this sitting at home but am hacking at the library at the moment. It is heavenly quiet here. I will post them later.


Kinda half-baked aside: Sometimes I think it would be incredibly valuable to share our intermediate steps when working out solutions to problems. Kinda like how it's fascinating to see PG write that essay.


So far I'm stumbling over the part where I actually sit down and read the book and do the problems. ;)

But my book only arrived last week, so I'm not feeling too guilty yet.


One function of this group is to turn guilt into shame. :)


The intro lecture on row/column pictures made this click for me better than the book did:

http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...

22:00 in.


How's everybody doing? I'm barely into 1.4, so I'm planning to work through a fair bit this weekend in order to be done the chapter a week from now. Should we discuss specific solutions from 1.2-1.3? Did anybody do the exercise marked "very optional" (1.3.29)? Do we need to discuss ass-kicking tactics to keep ourselves on track?


I'm mostly through 1.2 but not on 1.3 yet.

Paging just basic algebra back in was more annoying than I expected it to be (highly recommend for those like me that you grab some cheat sheets; most of the exercises in 1.2 are very straightforward if you have systems of equations in L1 head cache).

I'll try my best to keep up with you. I'm in NYC next week but I can kill some hotel downtime with this too.

Very yes, regarding ass-kicking tactics.


It seems unlikely that we're going to hit the target of having Chapter 1 done a week from today, especially for anyone who wants to do all the exercises. Time to recalibrate?

Ass-kicking makes sense only if there are slack resources that haven't been allocated to this, but could be. Is that your case? What do you think would make a difference?

One idea that occurred to me is a buddy system wherein two people would agree to nag each other and perhaps review answers to exercises, or at least confirm that they were done. Another is to leverage public commitment somehow. If I promise to do something, and don't do it, I feel bad.

Let's not forget the power of the forces we're up against. :)


I'm through the first 16 exercises of the 1.3 problems. Probably won't find time for the rest for a couple of days.

We definitely want to make sure that everyone gets the "Recommended" problem from 1.3. It's a very important concept.


Do you mean 1.3.18, which asks why it is impossible for a system of linear equations to have exactly two solutions? This is one where people would probably come up with a variety of different explanations.

Algebraically, if X and Y are solutions, then aX + bY is a solution if a + b = 1.

Geometrically, if two planes intersect at a line, and you try to add a third plane that shares two points with them, you're forced to "capture" the entire line.

Comments and/or additional ways of looking at this?


Yes, that's the one I mean, and those explanations pretty much capture it.

I might phrase the first explanation more simply as: If X and Y are solutions, then the point halfway between X and Y (i.e. X/2 +Y/2) is a solution. Then you can deploy Zeno's "paradox" to conclude that there is in fact an entire line of solutions. ;) But, of course, your statement is the more general one.

This is important because it captures the vital quality of linear systems, the quality that makes linear algebra worth studying in the first place: Once you've got more than one solution to a linear system, any weighted sum of those solutions is also a solution. This leads to the exciting possibility that you don't need to carefully seek out every single solution (of which there is often an infinite number); you just need to find a handful of (hopefully simple) solutions that are sufficient to produce any of the other solutions by constructing weighted sums. This is pretty much the fundamental strategy of physics: Fourier analysis is the classic example of that strategy at work, but it's also the basis of quantum mechanics.


Fabulous comment. Makes me think you're sandbagging us with this "I never learned linear algebra" schtick :)

I was musing about this off and on yesterday and another way of looking at it struck me. Linear systems are flexible in some (well, few) ways and rigid in others. The reason you can't build a model that just captures two points and then absconds without getting stuck with infinitely more is that, being linear, this stuff can't "bend". Another way of putting it (the sort of thing someone who has spent too much time around computing and not enough around geometry would say) is that the language of linear systems is expressive enough to express certain things but not others. The possible interactions are regimented. You can add dimensions to gain degrees of freedom, or equations to collapse them, but both "freedom" and "restriction" here have a pretty narrow meaning. I'm looking forward to learning about some of the magic things this theory can be made to do. It makes sense that something with such a regular, yet complex structure (complex by our standards anyway - mathematicians would no doubt chuckle) could yield some beautiful results, but that it found such a variety of real applications is surprising to me. More surprising in this respect than its older cousin, calculus, which starts off much closer to our naive sense of how reality works.

Now for those matrix exercises.


I was supposed to be in NYC this week but pushed it out, so I think if you're going to be done with 1.3 problems in a couple days, I stand a good chance at catching up to you.


Clickable:

http://www.hnsearch.com/search#request/all&q=%22linear+a...


Has anyone tried the "very optional" exercise (1.3.29)?


Yes, it was somewhat interesting though it does seem to me basically off-topic.

My answer is here, if it's helpful: http://pastebin.com/aPB1v2j9


Good for you! I haven't decided yet if I want to do that one. It seemed tangential to me too.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: